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Fiscal  Years  1990  through  1998,  maintenance  error  was  a  causal  factor  in  approximately 
one  out  of  every  five  Class  A  Flight  Mishaps.  Presently  there  is  an  on-going  effort  to 
identify  and  systematically  reduce  factors  contributing  to  human  error  in  Naval  Aviation 
maintenance.  This  study  administers  Baker’s  (1998)  Maintenance  Climate  Assessment 
Survey  (MCAS),  which  evaluates  factors  contributing  to  high  reliability,  to  nearly  1000 
participants  from  the  Naval  Fleet  Logistics  Support  Wing  (FLSW).  The  purpose  of  this 
study  is  to  assess  maintainer  perspectives  of  maintenance  operations  and  safety  culture 
within  their  respective  communities.  This  study  finds  statistically  differentiable 
responses  among  the  aircraft  communities  that  comprise  the  FLSW;  differences  that 
potentially  will  help  in  identifying  and  developing  intervention  strategies  to  further 
reduce  human  error  in  aviation  maintenance.  Additionally,  a  proposed  list  of  MCAS 
questions  is  produced  for  fleet  wide  distribution 
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EXECUTIVE  SUMMARY 


Despite  Naval  Aviation’s  success  in  cutting  its  Class  A  Flight  Mishap  (FM)  rate 
in  half  each  successive  decade  between  1950  and  1990,  the  number  of  aircraft  losses 
attributable  to  human  error  has  remained  relatively  constant  in  the  last  decade.  In 
attempting  to  meet  its  stated  goal  of  cutting  the  Naval  Aviation’s  Class  A  FM  rate 
attributed  to  human  factors  in  half  by  FY2000,  the  Human  Factors  Quality  Management 
Board’s  (HFQMB)  initial  focus  on  aircrew  error  has  generated  positive  results.  Since  the 
HFQMB’s  inception  in  1996,  the  U.S.  Navy  Class  A  FM  rate  dropped  to  its  lowest  point 
in  FY  1997  and  the  U.S.  Marine  Corps  posted  its  lowest  rate  in  FY  1998.  Unfortunately, 
the  singular  focus  on  reducing  aircrew  error  has  failed  to  attain  the  overall  goal.  Over  the 
past  decade,  approximately  one  in  five  Naval  Aviation  Class  A  FMs  were  determined  to 
have  maintenance  error  as  a  casual  factor.  The  HFQMB’s  prior  success,  coupled  with  a 
Naval  Air  Systems  Command  drive  to  address  ongoing  aircraft  maintenance  issues,  has 
compelled  the  HFQMB  to  further  analyze  aviation  maintenance  in  order  to  assist  in 
reaching  its  stated  goal. 

The  HFQMB  is  currently  shifting  gears  to  aviation  maintenance  and  has 
established  a  Human  Factors  in  Maintenance  and  Material  (HFMM)  Process  Action 
Team  (PAT).  The  HFMM  PAT  adapted  the  HFQMB’s  three-pronged  approach  to  further 
analyze  maintenance-related  errors  and  to  find  systems  or  processes  to  eliminate  them. 
The  three-pronged  approach  included  extensive  mishap  data  analysis,  benchmarking,  and 
a  climate  assessment  survey.  The  PAT’s  initial  concentration  was  on  major  Class  A 
mishaps  and  common  latent  conditions  that  contributed  to  the  active  failures  that  caused 
major  Maintenance  Related  Mishaps  (MRM).  Schmorrow  applied  a  maintenance 
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extension  to  the  Human  Factors  Analysis  and  Classification  System  (HFACS)  while 
examining  470  Naval  Aviation  MRMs  in  order  to  identify  the  most  likely  forms  of 
maintenance  error  and  the  conditions  associated  with  those  errors.  The  HFACS 
maintenance  extension  categories  of  error—  squadron,  violation,  unforeseen,  and  crew 
resource  management  -  were  found  to  be  the  most  significant  contributors  in  terms  of 
cost.  In  analyzing  124  Naval  Aviation  incidents,  which  occurred  during  FY90-FY98, 
Teeters  discovers  two  initial  areas  for  immediate  consideration:  contractor  related 
incidents  and  those  incidents  with  procedural  violations.  The  PAT  also  benchmarked 
other  organizations  and  programs,  which  soon  led  to  the  creation  and  implementation  of 
the  Groundcrew  Coordination  Training  (GCT).  Lastly,  the  Maintenance  Climate 
Assessment  Survey  (MCAS)  was  created  and  implemented  in  order  to  gain  valuable 
insight  into  the  maintenance  community’s  perception  of  its  safety  climate  and  culture. 

This  thesis  is  the  final  piece  of  the  mishap  data  analysis,  benchmarking,  and 
climate  assessment  tripod  currently  being  conducted  at  the  request  of  the  Commander, 
Fleet  Logistic  Support  Wing.  The  study  has  nearly  1000  participants  from  three  different 
aircraft  communities  that  comprise  the  FLSW.  The  results  of  this  study  conclude  that  the 
MCAS  can  be  utilized  as  a  tool  for  effectively  capturing  an  aviation  maintained  s 
perceptions  of  safety  in  maintenance  operations.  Through  analysis  of  the  MCAS  it  is 
found  that  the  components  of  Cummunication/Functional  Relationships  and  Risk 
Management  pose  the  most  immediate  concern  to  the  aviation  maintained  within  the 
FLSW.  Thus,  these  areas  would  serve  as  good  starting  points  for  mitigating  risk  in 
aviation  maintenance  operations.  Overall,  the  general  safety  climate  of  the  FLSW  is 
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assessed  to  be  good;  however,  some  potential  areas  within  each  community  have  been 
identified  for  focusing  and  prioritizing  safety  intervention  efforts. 

Through  Analysis  of  Variance  (ANOVA)  and  Multiple  Comparison  testing  this 
study  shows  that  the  mean  responses  of  the  C-9  community  are  statistically  different  than 
the  mean  responses  of  both  the  C-130  and  C-20  aircraft  communities  that  comprise  the 
FLSW  based  upon  a  model  of  High-Reliability  Organizations  (HROs).  Some  generalized 
observations  can  be  made  in  analyzing  the  Model  of  Organizational  Safety  Effectiveness 
(MOSE)  component  mean  responses.  Almost  across  the  board  the  C-9  community’s 
mean  responses  to  questions  are  noticeably  higher  than  those  of  both  the  C-130  and  C-20 
communities.  Moreover,  the  culture  of  the  C-130  maintenance  community  is  very 
sensitive  to  questions  that  pertain  to  issues  of  being  over-committed,  under¬ 
manned/under-staffed,  burdening  disproportionate  workloads,  the  eroding  effects  of 
personnel  turnover,  and  the  negative  impact  of  collateral  duties  acting  upon  maintenance 
safety.  The  old  adage  of  accomplishing  more  with  less  seems  to  have  worn  thin  within 
the  C-130  maintenance  community.  Additionally,  the  C-20  maintenance  community 
seems  to  express  concern  with  coordination,  functional  relationships,  and  communication 
related  issues.  Lastly,  statistical  significance  is  also  found  along  the  MOSE  components 
which  may  also  assist  in  helping  to  prioritize  intervention  efforts  within  the  FLSW. 

Naval  Aviation  does  not  need  to  wait  until  another  aircraft  mishap  occurs  until  it 
attempts  to  understand  what  is  going  on  within  the  safety  climate/culture  of  its  squadrons. 
The  MCAS  is  best  suited  to  be  utilized  as  a  proactive  tool  by  Squadron  Commanders  and 
Aviation  Safety  Officers  (ASOs)  to  identify,  prioritize,  and  focus  their  safety  intervention 
efforts  in  order  to  continue  to  reduce  human  error  in  aviation  maintenance  operations. 
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Lastly,  a  proposed  revised  set  of  MCAS  questions  is  enclosed  for  future  use  by 
Commanders  and  ASOs  as  a  tool  to  identify  the  most  urgent  process  problems  in  ongoing 
aviation  maintenance  operations. 
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INTRODUCTION 


Despite  Naval  Aviation’s  success  in  cutting  its  Class  A  Flight  Mishap  (FM)  rate 
in  half  each  successive  decade  between  1950  and  1990  (NPS,  School  of  Aviation  Safety, 
1998),  the  proportion  of  aircraft  losses  attributable  to  human  error  has  remained  relatively 
constant  in  recent  years  (Nutwell  &  Sherman,  1997).  Following  the  tragic  1996 
Nashville,  TN  F-14  mishap,  senior  Naval  Leadership  established  a  Human  Factors 
Quality  Management  Board  (HFQMB)  with  a  goal  of  cutting  Naval  Aviation’s  Class  A 
FM  rate  attributed  to  human  error  in  half  by  Fiscal  Year  (FY)  2000  (HFQMB,  1997).  In 
an  attempt  to  meet  this  goal,  the  HFQMB  focused  initially  on  aircrew  error,  which  was 
found  to  be  a  causal  factor  in  three  out  of  every  five  Class  A  FMs.  Using  an  approach 
similar  to  that  in  investigating  aircrew  error,  the  HFQMB  have  since  expanded  their  focus 
to  investigating  human  factors  maintenance-related  errors.  This  thesis  contributes  to  this 
investigation  by  capturing  the  maintainers’  perspectives  of  risk  management,  safety 
climate  and  culture  within  the  aviation  maintenance  community. 

A.  OVERVIEW 

The  HFQMB ’s  (1997)  strategy  consists  of  concentrated  effort  in  three  areas  of 
evaluation:  (1)  Mishap  Data  Analysis,  (2)  Organizational  Benchmarking,  and  (3) 
Command  Safety  Assessment.  The  objective  of  analyzing  mishap  data  is  to  identify  the 
human  factors  issues  in  past  Class  A  FM  cases.  The  HFQMB  identify  both  aircrew  and 
supervisory  error  as  major  contributing  factors.  The  subsequent  analysis  allowed  the 
HFQMB  to  prioritize  the  intervention  target  areas  based  upon  prevaling  human  errors 
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contributing  most  to  FMs.  The  benchmarking  effort  uncovers  the  best  practices  and 
processes  utilized  in  other  aviation  organizations  such  as  the  commercial  airlines  use  of 
line  oriented  flight  training.  In  short  order  the  Aircrew  Coordination  Training  (ACT) 
program  was  being  targeted  to  give  aircrews  a  means  to  perform  Computer  Aided 
Debriefs  (CADs)  following  aircrew  simulation  training  (Nutwell  &  Sherman,  1997). 
Finally,  a  Command  Safety  Assessment  Survey  (CS  AS),  based  on  a  model  of  high 
reliability  organizations  (Libuser  1994;  Roberts  1990)  was  developed  to  assess  a 
command’s  safety  climate  from  an  aircrew  perspective  (Ciavarelli  &  Figlock,  1997).  The 
survey’s  results  indicated  that  organizational  and  supervisory  issues  were  seen  as 
impacting  flight  safety  (Nutwell  &  Sherman,  1997).  Since  the  HFQMB’s  inception  the 
U.S.  Navy  Class  A  FM  rate  dropped  to  its  lowest  point  in  Fiscal  Year  1997  (Naval  Safety 
Center,  1997)  and  the  U.S.  Marine  Corps  posted  its  lowest  flight  mishap  rate  in  Fiscal 
Year  1998  (Naval  Safety  Center,  1998).  These  advances  have  been  attributed  to  the 
HFQMB’s  primary  focus  and  efforts  to  reduce  aircrew  error,  however  the  proportion  of 
mishaps  attributable  to  human  error  has  remained  relatively  the  same  (Naval  Safety 
Center,  1999). 

In  early  1999,  the  Vice  Chief  of  Naval  Operations  reaffirmed  the  drive  to 
systematically  reduce  human  error  and  established  a  new  objective  of  reducing  related 
mishaps  from  current  levels  by  50  percent  at  the  end  of  FY2000  (T.  Meyers  personal 
communication,  January  8,  1999).  However,  only  limited  success  in  reducing  human 
error  related  aviation  mishaps  can  be  expected  if  only  aircrew  error  is  being  addressed. 
The  HFQMB’s  success,  coupled  with  a  Naval  Air  Systems  Command  drive  to  address 


2 


ongoing  aircraft  maintenance  issues  (Lockhardt,  1997),  compelled  it  to  start  examining 
the  maintenance  factors  that  lead  to  one  out  of  every  five  Class  A  FMs. 

The  HFQMB,  in  expanding  their  focus  to  include  aviation  maintenance,  formed  a 
Human  Factors  in  Maintenance  and  Material  (HFMM)  Process  Action  Team  (PAT).  The 
HFMM  PAT  elected  to  adapt  the  same  three-pronged  approach  utilized  in  examining 
aircrew  factors;  and  its  initial  concentration  was  on  63  major  maintenance-related  Class 
A  mishaps  and  the  common  latent  conditions  that  “set  the  stage”  for  active  failures  that 
cause  them  (Schmidt,  Schmorrow  &  Hardee,  1998).  The  results  of  their  study  indicate 
that  supervisory  and  climate/cultural  issues  play  a  direct  role  in  maintenance-related 
mishaps.  Schmorrow  (1998)  then  studies  470  Naval  Aviation  MRMs  ranging  from 
mishaps  to  more  routine  incidents  in  order  to  identify  the  most  common  forms  of 
maintenance  error  and  the  conditions  associated  with  them.  Utilizing  the  same  analytical 
process  he  again  found  that  supervisory  and  climate/cultural  issues  play  a  significant  role 
in  maintenance  error. 

Early  maintenance  benchmarking  initiatives  uncovered  a  maintenance  resource 
management  program  for  maintainers  similar  to  the  cockpit  resource  management 
program  for  aircrew.  A  Naval  Aviation  Groundcrew  Coordination  Training  (GCT) 
program  was  developed  and  tested  producing  promising  results  (Sian,  1997).  The  GCT 
proved  to  be  a  meaningful  training  program  in  which  participants  learned,  as  measured  by 
a  content-knowledge  test,  valuable  information  that  affected  real  changes  in  behavior  and 
attitudes.  Additionally,  a  three-month  follow-up  to  the  GCT  training  indicated  a  decrease 
in  unsafe  behavior  exhibited  by  GCT  participants. 
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Finally,  Baker  (1998)  develops  a  Maintenance  Climate  Assessment  Survey 
(MCAS)  based  upon  a  model  of  high  reliability  organizations  in  an  attempt  to  capture  the 
maintainers’  perspective  of  risk  management,  safety  climate  and  culture  within  the 
aviation  maintenance  community.  After  first  developing  a  survey  to  capture  aviation 
maintainers’  attitudes  towards  maintenance  operations,  he  then  successfully  reduced  the 
prototype  67  question  survey  to  a  comprehensive  35  question  instrument.  This  35 
question  variant  of  the  MCAS  is  applied  in  this  study  to  identify  trends  and  possible 
intervention  opportunities  in  an  effort  to  further  reduce  human  error  in  aviation 
maintenance  operations  within  the  Navy’s  Fleet  Logistics  Support  Wing.  This  study  also 
parallels  concurrent  research  being  conducted  by  Oneto  (1999),  that  concentrates  on 
safety  postures  across  differing  aircraft  communities  within  a  common  Naval  Reserve 
Wing. 

B.  BACKGROUND 

The  Fleet  Logistic  Support  Wing  (FLSW)  is  comprised  of  fourteen  squadrons 
from  three  very  different  aircraft  communities:  the  C-9B  “Skytrain  II,”  the  C-130T 
“Hercules,”  and  the  C-20  “Gulfstream”  that  cumulatively  fly  over  62,000  hours  a  year  all 
over  the  world.  The  27  C-9Bs  are  the  high-speed  workhorses  of  the  FLSW  with  a 
carrying  capacity  of  90  passengers  or  28,000  pounds  of  cargo  at  ranges  of  up  to  2,300 
miles.  Each  of  the  20  heavy-lift  transport  C-130s  have  a  4,100  mile  range  and  can 
transport  45,000  pounds  of  cargo  or  up  to  75  passengers  to  even  the  most  austere 
airstrips.  Lastly,  for  high  speed/priority  deliveries  there  are  6  C-20s  with  carrying 
capacities  of  4,500  pounds  of  cargo  or  26  people  at  ranges  of  up  to  5,200  miles  (Peniston, 
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1998).  The  U.S.  Naval  Reserve  FLSW  is  a  vital  component  of  the  worldwide  logistics 
pipeline  that  transports  Navy  and  Marine  Corps  personnel  and  parts  worldwide  and 
resupplies  Naval  military  forces  in  major  supply  centers  and  smaller  airfields,  alike 
(Peniston,  1998).  Given  decreasing  budgets,  force  reductions,  reduced  spare  part 
inventories  and  the  evolving  concept  of  "precision  logistics",  a  potential  Class  A  FM  or 
any  other  type  or  severity  of  incident  within  the  FLSW  could  have  detrimental  effects  on 
forward  deployed  forces’  abilities  to  fulfill  their  given  missions. 

The  Commander  of  the  FLSW  became  aware  of  the  HFMM’s  recent  efforts  and 
requested  a  systematic  assessment  of  the  Wing’s  maintenance  safety  posture.  The  Naval 
Postgraduate  School’s  School  of  Aviation  Safety  became  engaged  in  a  number  of  efforts 
to  assist  in  this  process.  Efforts  included  the  Human  Factors  Analysis  and  Classification 
System  study  of  124  MRMs  (Teeters,  1999),  GCT  briefs  at  each  Squadron  location  and 
follow  up  workshops,  maintenance  oriented  operational  risk  management  training,  and 
the  Wing  wide  administration  of  the  MCAS.  Additionally,  the  results  of  precious  Naval 
Safety  Center  on-site  maintenance  safety  surveys  were  also  aggregated.  The  FLSW 
initiative  was  culminated  with  a  maintenance  human  factors/risk  management  workshop 
hosted  by  the  School  of  Aviation  Safety  to  discuss  the  results  from  each  effort.  The 
findings  were  used  to  determine  common/prevalent  problem  areas,  prioritizing  them 
according  to  severity/probability,  and  to  develop  notional  intervention  strategies. 

C.  PROBLEM  STATEMENT 

The  Navy’s  Fleet  Logistics  Support  Wing  has  an  outstanding  mishap  record  with 
only  one  Class  A  mishap  in  the  last  20  years.  However,  maintenance  factors  continue  to 


5 


be  a  factor  in  one  out  of  every  five  Naval  Aviation  mishaps.  This  thesis  constitutes  the 
final  prong  of  the  HFMM’s  three-pronged  approach  currently  being  conducted  at  the 
request  of  the  Commander,  FLSW.  The  study  utilizes  the  35  question  variant  of  the 
MCAS  in  order  to  capture  and  assess  the  organizational  safety  climate  of  the  FLSW’s 
aviation  maintenance  community  based  upon  the  model  of  high  reliability  organizations 
in  order  to  help  identify  and  reduce  human  error  in  Naval  Aviation.  Accordingly,  it 
investigates  the  following  questions: 

1.  What  is  the  safety  climate  of  the  FLSW  based  upon  a  model  of 

“high  reliability”  organizations  from  the  aircraft  maintained  perspective? 

2.  Are  there  discemable  differences  between  the  three  aircraft  communities 
surveyed  that  comprise  the  FLSW  based  upon  the  “high  reliability” 
organization  model? 

3.  Do  the  results  of  the  survey  match  perceptions  of  actual  errors  and  factors  that 
lead  up  to  them? 

4.  Can  the  Maintenance  Climate  Assessment  Survey  (MCAS)  be  improved? 

D.  SCOPE  AND  LIMITATIONS 

The  intent  of  this  study  is  to  gain  a  better  insight  into  the  climate  and  culture  of 
the  maintenance  community  of  the  FLSW  in  order  to  better  develop  intervention 
strategies  that  would  help  reduce  human  error  in  aviation  maintenance.  It  is  hoped  that 
lessons  learned  through  this  study  can  produce  procedures  and  processes  that  are  both 
directly  applicable  and  beneficial  to  all  of  Naval  Aviation.  For  the  purpose  of  this  study 
only  Naval  Aviation  maintenance  personnel  of  the  squadrons  that  comprise  the  FLSW  are 
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surveyed.  This  study  includes  13  of  the  14  FLSW  squadrons;  the  lone  squadron  not 
surveyed  (VR-51,  Kaneoe  Bay,  HI)  utilizes  civilian  contract  maintenance  of  its  aircraft. 

The  survey  was  administered  on  site  and  in  a  group  setting  at  the  various 
participating  squadrons.  Additionally,  the  survey  was  given  in  conjunction  with  a 
scheduled  maintenance  safety  presentation  on  human  factors  issues  in  aviation.  The 
squadrons  were  in  various  stages  of  training  and  operational  tasking  at  the  time  of  the 
survey  being  administered.  Some  Squadrons  were  in  a  safety  standdown,  some  were 
taken  during  a  “drill”  weekend  while  still  other  Squadrons  were  in  “off-drill”  weekend. 
Lastly,  some  were  executing  a  normal  flight  schedule  routine  while  other  Squadrons 
where  in  a  “no  fly”  status  weekend.  The  varying  operational  taskings  that  the  Squadrons 
were  simultaneously  undertaking  during  the  administration  of  the  MCAS  accounted  for 
much  of  the  variance  in  the  number  of  surveys  collected  from  each  Squadron.  The 
potential  MCAS  respondents  were  briefed  on  the  survey  and  its  purpose  and  questions 
that  arose  pertaining  to  the  survey  were  answered  by  the  survey  administer.  The  surveys 
were  then  immediately  collected  upon  completion  to  allow  for  maximum  accountability. 
For  the  purpose  of  this  study,  two  squadron’s  responses  (VR-55  and  VR-57)  were 
transformed  from  the  67  item  MCAS  utilizing  the  mapping  indicated  in  Baker’s  (1998) 
thesis  to  the  current  35  item  MCAS. 

Chapter  II  contains  a  literature  review  on  investigative  scope,  accident  causation, 
climate,  culture  and  safety,  and  high  reliability  organizations.  In  chapter  III  the 
methodology  used  in  this  study  is  discussed.  Chapter  IV  presents  the  results  of  this 
study.  Lastly,  chapter  V’s  conclusions  include  findings  and  recommendations. 
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E.  DEFINITIONS 


This  study  uses  the  following  definitions  (DON,  1989): 

Naval  Aircraft.  Refers  to  U.S.  Navy,  Naval  Reserve,  U.S.  Marine  Corps,  and  U.S. 
Marine  Corps  Reserve  aircraft. 

Mishap.  A  Naval  Aviation  mishap  is  an  unplanned  event  or  series  of  events  directly 
involving  naval  aircraft,  which  result  in  $10,000  or  greater  cumulative  damage  to  naval 
aircraft  or  personnel  injury. 

Mishap  Class.  Mishap  severity  classes  are  based  on  property  damage  and  personnel 
injury.  The  following  are  the  definitions  of  the  three  major  severity  classes: 

a.  Class  A.  A  mishap  in  which  the  total  cost  of  property  damage  (including  all 
aircraft  damage)  is  at  least  $1,000,000;  or  a  naval  aircraft  is  destroyed  or  missing;  or  any 
fatality  or  permanent  total  disability  of  a  person  occurs  with  direct  involvement  of  naval 
aircraft. 

b.  Class  B.  A  mishap  in  which  the  total  cost  of  property  damage  (including  all 
aircraft  damage)  is  between  $200,000  and  $1,000,000  and/or  a  permanent  partial 
disability  occurs  and  or  five  or  more  personnel  are  hospitalized. 

c.  Class  C.  A  mishap  in  which  the  total  cost  of  property  damage  (including  all 
aircraft  damage)  is  between  $10,000  and  $200,000  and/or  injury  results  in  one  or  more 
lost  workdays. 
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II.  LITERATURE  REVIEW 


A.  OVERVIEW 

In  April  1996,  Vice  Admiral  Bennitt  established  the  Human  Factors  Quality 
Management  Board  (HFQMB)  in  order  to  address  the  biggest  remaining  challenge  in 
aviation  safety-  preventing  human  error.  To  aid  in  matching  accident  causal  factors  to 
appropriate  intervention  areas  the  HFQMB  is  using  a  model  for  human  error  created  by 
the  Naval  Safety  Center  (Nutwell  &  Sherman,  1997).  Human  Factors  Analysis  and 
Classification  System  (HFACS),  an  adaptation  of  Reason’s  (1990)  human  error  model,  is 
applied  to  classify  Naval  aircraft  mishaps.  Early  HFACS  analysis  identified  classic 
human  error  forms  described  by  Reason  such  as  slips,  lapses,  mistakes,  and  violations. 
HFACS  also  identified  human  errors  related  to  supervisory  and  aircrew  conditions  that 
“set  the  stage”  for  active  human  error  forms  to  occur.  (Nutwell  &  Sherman,  1997).  The 
application  of  HFACS  to  Fiscal  Years  1990-1996  Class  A  Naval  Aviation  FMs  indicated 
that  supervisory  factors  could  be  identified  in  over  half  of  the  so-called  “pilot  error” 
mishaps  (Ciavarelli  &  Figlock,  1998). 

Of  the  three  kinds  of  human  activity  that  are  universal  in  hazardous  technologies 
such  as  Naval  Aviation:  control  under  normal  conditions,  control  under  emergency 
conditions,  and  maintenance-related  activities.  Reason  (1997)  asserts  that  the  latter  of  the 
three  poses  the  largest  human  factors  problem.  He  contends  “Errors  by  pilots,  control 
room  operators,  and  other  ‘sharp-end’  personnel  may  add  the  finishing  touches  to  an 
organizational  accident,  but  it  is  often  latent  conditions  created  by  maintenance  lapses 
that  either  set  the  accident  sequence  in  motion  or  thwart  its  recovery.” 
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The  Maintenance  Extension  (ME)  of  HFACS  as  developed  by  Schmidt, 
Schmorrow,  and  Hardee  (1998)  depicts  Supervisory,  Maintainer  and  Working  Conditions 
as  latent  factors  that  can  impact  a  maintainer’ s  performance  and  can  contribute  to  an 
active  failure,  an  Unsafe  Maintainer  Act.  An  unsafe  maintainer  act  may  immediately  or 
eventually  lead  directly  to  a  mishap  or  injury.  Additionally,  the  unsafe  maintainer  act 
may  contribute  to  and  become  a  latent  maintenance  condition  to  be  consequently  dealt 
with  by  “sharp  end”  personnel  such  as  aircrew  (see  Figure  1).  The  HFACS  ME  depicts 
the  inpact  of  organizational  climate  on  maintenance  safety. 


Figure  1.  HFACS  Maintenance  Extension  Diagram. 

B.  INVESTIGATIVE  SCOPE 

A  number  of  experts  within  the  aviation  research  community  have  recently  argued 
that  there  is  now  an  urgent  need  to  complement  analyses  of  individual  human  error  by 
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moving  towards  an  understanding  of  the  role  played  by  broader  system  factors  in 
accidents  such  as  organizational  factors.  The  International  Civil  Aviation  Organization 
(ICAO)  states  that: 

The  late  70’s,  80’s,  and  90’s  will  undoubtedly  be  remembered  as  the 
golden  era  of  aviation  Human  Factors.  Cockpit  (and  then  Crew) 

Resource  Management  (CRM),  Line-Oriented  Flight  Training  (LOFT), 

Human  Factors  training  programs,  attitude-development  programs  and 
similar  efforts  have  multiplied,  and  a  sustained  campaign  to  increase  the 
awareness  of  human  error  in  aviation  safety  has  been  initiated.  But 
much  to  the  consternation  of  safety  practitioners  and  the  entire  aviation 
community,  human  error  continues  to  be  at  the  forefront  of  accident 
statistics,  (p.  1) 

The  ICAO  authors  go  on  to  point  out  how  human  error  is  often  precipitated  by  more 
systematic,  background  management  and  organizational  factors. 

Adams  and  Payne  (1992)  further  underscore  the  point  that  only  limited  success  in 
reducing  pilot-error  accident  rates  will  be  achieved  if  the  pilot  is  the  only  part  of  the 
operational  problem  being  fixed.  Enders  (1992)  makes  the  additional  observation  that 
most  aviation  accidents  have  several  causes  and  that  to  seek  a  single  “probable”  cause  of 
an  event,  such  as  pilot  error  or  maintenance  error  therefore  misses  opportunities  for 
learning.  He  further  suggests  that  causes  involving  “management  or  supervisory 
inattention  at  all  levels”  are  the  most  prevalent  category,  and  perhaps  contribute  as  much 
to  accidents  as  the  total  numbers  of  pilot  and  maintenance  errors  put  together.  Johnston 
(1991)  took  Enders’  one  step  further  in  asserting  that  the  need  for  aircraft  accident 
analysts  to  adopt  a  wider  investigative  reality.  For  example,  for  those  instances  in  which 
individuals  are  found  to  have  failed  to  follow  Standard  Operating  Procedures,  any 
conclusion  of  the  type  “the  accident  occurred  because  X  failed  to  follow  procedure  Y” 
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should  not  simply  be  the  end  of  the  investigation,  but  rather  should  always  be  followed 
by  the  more  probing  question  of  “and  why  was  this  so?”  (Johnston,  1991). 

In  Naval  Aviation  and  elsewhere,  human  error  is  one  of  a  long-established  list  of 
"causes"  used  by  the  press  and  accident  investigators.  Reason  (1997)  argues  that  human 
error  is  a  consequence,  and  not  a  cause,  due  to  the  fact  that  human  errors  are  shaped  and 
provoked  by  precedent  workplace  and  organizational  factors.  He  explains  only  by 
understanding  the  context  in  which  an  error  was  provoked  can  investigators  hope  to 
prevent  the  error's  recurrence  explains  Reason.  There  are  several  stages  in  the 
development  and  investigation  of  an  organizational  accident  (see  Figure  2).  The 
triangular  shape  represents  the  system  producing  an  organizational  accident,  and  it  has 
three  levels:  the  person  (unsafe  acts),  the  workplace  (error-provoking  conditions),  and  the 
organization.  The  dark  upward  arrows  indicate  the  direction  of  causality  and  the 
downward  arrow  indicates  the  investigative  steps  (Reason,  1997). 


Figure  2.  Reason’s  Stages  in  Organizational  Accident  Development/Investigation. 
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Clearly,  one  can  begin  to  sense  the  rising  ground  swell  of  researchers  and  analysts 
seeking  a  new  way  of  conceptualizing  and  quantifying  processes  of  risk  handling  and 
management  in  regards  to  organizational  factors  in  aviation.  HFACS  ME  is  one  such 
tool  that  is  currently  being  explored  to  better  capture  and  conceptualize  those  latent 
workplace  and  organizational  factors  that  contribute  to  human  error.  A  more  in-depth 
understanding  of  these  factors  may  proactively  lead  to  intervention  measures  that  prevent 
such  accidents  from  ever  occurring. 

C.  ACCIDENT  CAUSATION 

Failures  in  large-scale  technological  systems  such  as  aviation  cannot  be  described 
in  technical  terms  alone.  Behavioral  causes  are  often  predominant  in  disasters  (Pidgeon 
&  O’Leary,  1994).  Many  of  the  behavioral  causes  of  disasters  can  be  traced  to  the  social 
and  organizational  arrangements  of  the  socio-technical  systems  associated  with  the  large- 
scale  hazards.  Pidgeon  and  O’Leary  (1994)  explain  that  the  concept  of  a  socio-technical 
system  stresses  the  close  inter-dependence  between  a  technology  and  the  human 
resources  necessary  for  its  use.  These  social  and  technical  components  interact  with,  and 
over  time  change  each  other  in  complex  and  often  unforeseen  ways  they  further  explain. 

It  is  now  clear  that  the  human  causes  of  disasters  in  complex  socio-technical  systems 
cannot  all  be  adequately  classified  under  the  catch-all  term  of  “human  error”  (Pidgeon, 
1991).  The  behavior  contributions  to  disasters  are  often  more  subtle  and  diverse  in  nature 
ranging  from  Reason’s  (1990)  slips,  lapses,  mistakes  and  faulty  decision  making  of 
individuals  to  more  systematic  and  complex  organizational  influences. 
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The  first  comprehensive  analysis  of  the  social  and  organizational  preconditions  to 
disaster  in  large-scale  technological  systems  is  by  Turner  (1978)  in  Man-Made  Disasters. 
As  a  result  of  detailed  analysis  of  84  major  accidents  in  the  United  Kingdom  over  a  ten- 
year  period  he  concludes  that  such  events  rarely  come  about  for  any  single  reason. 
Turner’s  (1978)  disaster  development  model  focuses  in  particular  upon  the  information 
difficulties  associated  with  the  attempts  of  individuals  and  organizations  to  deal  with 
uncertain  and  ill-structured  safety  problems.  He  further  concludes  that  prior  to  any 
disaster  it  is  typical  to  find  that  a  number  of  undesirable  events  accumulate  unnoticed  or 
not  fully  understood,  often  over  a  considerable  number  of  years. 

Turner  (1978)  defines  this  gradual  development  of  preconditions  as  the  “disaster 
incubation  period.”  He  goes  on  to  state  that  this  period  is  one  in  which  interacting  sets  of 
events  build  up  and  are  resolved  either  by  taking  preventative  action  to  remove  one  or 
more  of  the  undesirable  events  or  are  concluded  by  a  “triggering  event”  as  a  final  critical 
error  or  a  slightly  abnormal  operating  condition.  The  “trigger  event”  results  in  a 
catastrophe  and  consequential  investigation  sheds  light  on  previously  unseen  background 
factors.  However,  caution  is  warranted  in  that  the  immediate  trigger  may  be  confused 
with  the  more  systemic  background  causes  of  a  disaster,  or  may  even  be  taken  to  be  the 
sole  cause.  Turner  states  that  a  further  implication  of  the  incubation  period  is  that  many 
incidents  are  distinguished  from  full-scale  accidents  only  by  the  absence  of  a  suitable 
trigger  event,  perhaps  due  to  the  intervention  of  chance  factors,  with  the  background  and 
latent  factors  remaining  constant.  Therefore  near-miss  incidents  can  and  perhaps  must  be 
interpreted  as  important  warning  signals. 
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Perrow  (1984)  provides  a  second  and  related  account  of  failures  in  socio-technical 


systems  in  his  book  Normal  Accidents.  Perrow’ s  approach  is  a  more  technical  one  that 
draws  upon  his  experiences  from  a  perspective  of  systems  engineering.  Perrow  identifies 
two  general  characteristics:  complexity  (a  system  can  either  be  complex  or  linear)  and 
coupling  (a  system’s  coupling  can  be  either  tight  or  loose).  Systems  such  as  nuclear 
energy  and  chemical  processing,  exhibit  both  high  complexity  and  tight  coupling.  These 
co-occurrences  pose  particular  control  difficulties.  Such  systems  have  a  small  tolerance 
for  deviation  and,  according  to  Perrow  (1984),  will  be  subject  almost  inevitably  to 
“normal”  accidents. 

An  organization’s  position  within  an  operational  “safety  space”  is  determined  by 
the  quality  of  the  processes  used  to  combat  its  operational  hazards  explains  Reason 
(1997).  There  will  be  some  positive  correlation  between  an  organization’s  position 
within  the  safety  space  and  the  number  of  incidents  that  befall  it.  Organizations  that  he 
finds  at  the  resistant  end  of  the  space  are  likely  to  suffer  fewer  incidents  within  a  given 
time  frame  than  those  organizations  at  the  vulnerable  end.  This  correlation,  however, 
will  never  be  perfect  due  to  chance.  Additionally,  Reason  (1997)  contends  that  when 
incident  rates  occur  at  a  very  low  level,  as  in  Naval  Aviation,  the  occurrence  or  non¬ 
occurrence  of  an  incident  during  a  given  time  frame  reveals  very  little  about  an 
organization’s  position  within  the  safety  space.  Figure  3  shows  how  hypothetical 
organizations  could  be  distributed  within  the  safety  space  at  any  one  time.  Reason  asserts 
their  positions  within  the  space  are  determined  by  their  intrinsic  resistance  or 
vulnerability  to  their  operating  hazards. 
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Figure  3.  Reason’s  (1997)  “safety  space.” 

D.  CLIMATE,  CULTURE,  AND  SAFETY 

Organizational  climate,  by  virtue  of  being  a  more  salient  phenomenon  of 
organizational  culture,  lends  itself  to  direct  observation  and  measurement  and  thus  has 
had  a  longer  research  tradition  (Schein,  1990).  Zohar  (1980)  discusses  various  concepts 
of  safety  climate.  She  considers  safety  climate  to  be  a  subset  of  an  organization’s  overall 
climate.  Safety  climate  refers  to  the  shared  perception  of  an  organization’s  members  that 
the  organization’s  leaders  are  genuinely  committed  to  safety  of  operations,  and  have 
taken  appropriate  measures  to  communicate  safety  principals,  and  to  ensure  adherence  to 
safety  standards  and  procedures  (Zohar,  1980).  Climate  is  only  a  surface  manifestation 
of  culture  suggests  Schein  (1990),  and  thus  research  on  climate  has  not  enabled 
researchers  to  delve  into  the  deeper  causal  aspects  of  how  organizations  function. 
Explanations  for  variations  in  climate  and  norms  are  essential,  and  it  is  the  need  for  this 
illusive  information  that  ultimately  drives  analysts  to  “deeper”  concepts  such  as  culture 
(Schein,  1990). 
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Current  interest  in  the  term  “safety  culture”  can  be  traced  directly  to  the  accident 
at  the  Chernobyl  nuclear  plant  in  the  former  Soviet  Union  and  the  response  of  the 
Western  nuclear  industries  to  the  human  and  organizational  causes  of  the  disaster 
(Pidgeon,  1991).  It  is  argued  that  safety  culture  represents  a  new  way  of  conceptualizing 
processes  of  risk  handling  and  management  in  organizational  and  other  contexts  (Turner, 
Pidgeon,  Blockley,  &  Toft,  1989).  Turner  et  al.  (1989)  broadly  define  safety  culture  as 
the  set  of  beliefs,  norms,  attitudes,  roles  and  social  and  technical  practices  within  an 
organization  which  are  concerned  with  minimizing  the  exposure  of  individuals,  both 
within  and  outside  of  the  organization,  to  conditions  considered  to  be  dangerous.  A 
safety  culture  is  more  than  a  group  of  individuals  enacting  a  set  of  safety  guidelines;  it  is 
a  group  of  individuals  guided  in  their  behavior  by  their  joint  belief  in  the  importance  of 
safety,  and  their  shared  understanding  that  every  member  willingly  upholds  the  group’s 
safety  norms  and  will  support  other  members  to  that  common  end  (Merritt  &  Helmreich, 
1996). 

Merritt  and  Helmreich  (1996)  explain  that  culture  gives  us  cues  and  clues  on  how 
to  behave  in  normal  and  novel  -situations,  thereby  making  the  world  less  uncertain  and 
unpredictable  for  us.  They  further  describe  two  important  and  distinct  components  of 
culture.  The  surface  culture,  or  outer  layer  of  culture  consists  of  observable  behaviors 
and  recognizable  physical  manifestations  such  as  members’  uniforms,  signs  and  logos, 
and  documents.  The  deep  structure,  or  inner  layer  of  culture,  consists  of  the  values, 
beliefs,  and  assumptions  that  underlie  the  surface  structure  and  provide  the  logic  that 
guides  the  members’  behaviors  (Merritt  &  Helmreich,  1996). 
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Schein  (1990),  in  his  comprehensive  review  of  organizational  culture  described 
how  culture  is  the  driving  force  and  the  guiding  principal  behind  an  organization’s  goal 
structure,  a  means  to  attaining  goals,  the  source  of  criteria  for  measuring  progress,  and 
the  origination  of  methods  for  correcting  deviations  from  norms  and  expected  outcomes. 
He  posits  once  an  organization  has  evolved  into  a  mature  culture  because  it  has  had  a 
long  and  rich  history,  that  culture  creates  the  patterns  of  perception,  thought,  and  feeling 
of  every  new  generation  in  the  organization  and,  therefore,  also  “causes”  the  organization 
to  be  predisposed  to  certain  kinds  of  leadership.  In  that  sense  the  mature  group,  through 
its  culture,  also  creates  it  own  leaders.  An  important  paradox  of  mature  culture  to 
understand,  explains  Schein  (1985)  is  this:  leaders  create  cultures,  but  cultures,  in  turn 
create  their  next  generation  of  leaders. 

Schein  (1990)  finds  culture  also  molds  behavior  of  individuals  through  a  system 
of  rewards,  expectations  about  status,  power,  authority,  established  group  boundaries  for 
inclusion  or  exclusion,  and  underlying  concepts  for  managing  deviations  from  norms.  He 
adds  that  culture  is  learned  by  individuals  who  join  an  organization  and  is  strongly 
influenced  by  the  organization's  structure  and  leadership.  The  principal  factors  that 
contribute  to  an  organization's  culture  include  operational  criteria  for  personnel  selection, 
formal  training  practices,  explicit  and  implicit  role  expectations,  and  especially  the 
actions  of  leaders  as  demonstrated  by  their  example.  Schein  (1990)  believes  an 
organization's  culture  is  heavily  influenced  by  what  leaders  pay  attention  to,  and  by  what 
they  express  as  the  core  values  or  expectations  of  personnel  under  their  supervision. 

Pidgeon  and  O’Leary  (1994)  view  the  principal  cultural  unit  within  which  a  safety 
culture  is  assumed  to  be  located  is  the  organization.  Within  the  organization,  members 
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will  all  be  selected,  trained,  and  work  within  a  corporate  setting  that  both  shapes  beliefs 
and  regulates  behavior.  They  feel  there  is  a  sense  in  which  it  is  misleading  to  talk  of  an 
organization  or  corporate  culture  per  se.  Rather,  it  is  possible  to  think  of  the  culture  of 
small  groups  of  workers  or  workcenter,  of  departments,  divisions,  and  organizations 
being  both  nested  within  and  sometimes  overlapping  one  another. 

E.  HIGH  RELIABILITY  ORGANIZATIONS 

Roberts  (1988)  and  Libuser  (1994)  explain  that  High-Reliability  Organizations 
(HROs),  organizations  that  operate  in  a  hazardous  environment,  yet  produce  very  low 
rate  of  accidents  and  incidents,  which  operate  effectively  and  safely  have  certain  key 
characteristics  in  common.  Some  of  those  characteristics  are  leadership  style,  sound 
safety  management  policies,  procedure  standardization,  adequacy  of  resources  and 
staffing,  a  defined  system  for  risk  management,  and  other  factors.  While  studying  on  site 
the  operations  of  "safe"  technologically  complex  organizations  such  as  the  U.S.  Navy’s 
nuclear  powered  aircraft  earners  Roberts  (1988)  argues  that  existing  organizational 
research  is  of  little  help  in  understanding  the  organizational  processes  within  such  High 
Reliability  Organizations  (HROs).  She  (1988)  claims  that  there  are  significant 
differences  in  the  internal  processes  of  organizations  as  a  function  of  the  degree  to  which 
their  production  technologies  are  perceived  as  hazardous  or  the  consequences  of 
individual  failures  vary  in  severity. 

Based  upon  the  earlier  research  of  Roberts,  Libuser  (1994)  developed  a  Model  of 
Organizational  Safety  Effectiveness  (MOSE)  that  categorized  the  traits  of  HROs  into  five 
components:  1)  Process  Auditing  (PA)  -  an  ongoing  system  to  monitor  hazards;  2) 
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Reward  System  (RS)-  expected  compensation  or  disciplinary  action  used  to  shape 
behavior;  3)  Quality  Assurance  (QA)-  policies  and  procedures  for  promoting  quality 
performance;  4)  Risk  Management  (RM)-  organizational  risk  perception  and  mitigation; 
and  5)  Command  and  Control.  (CC)-policies  and  procedures  to  mitigate  risks.  Examples 
of  conventional  organizations  requiring  and  possessing  high  reliability  today  are  power 
plants,  chemical  manufacturing  plants,  oil  refineries,  and  transportation  industries. 

The  elements  described  by  Libuser  and  Roberts  map  very  neatly  into  Turner  (et 
al.,  1989)’s  original  theoretical  account  of  what  might  constitute  a  good  safety  culture. 
Those  four  principal  areas  Turner  et  al.  propose  are:  1)  location  of  responsibility  for 
safety  at  strategic  management  level,  2)  distributed  attitudes  of  care  and  concern 
throughout  an  organization,  3)  appropriate  norms  and  rules  for  handling  hazards,  and  4) 
on-going  reflection  upon  safety  practice.  Additionally,  Degani  and  Weiner  (1994)  find 
there  is  a  need  to  have  alignment  between  all  levels  within  a  safety  culture:  philosophy 
(strategic),  policy  (tactical),  procedures  (operational),  and  practices  (working).  Schein 
(1985,  p.  2)  who  was  emphatic  about  the  primacy  of  leadership  influence  on  the  culture 
of  an  organization  would  argue  that  the  leaders  must  burden  the  responsibility  of  seeing 
that  alignment  earned  out  at  all  levels:  “The  only  thing  of  real  importance  that  leaders  do 
is  to  create  and  manage  culture  and  that  the  unique  talent  of  leaders  is  their  ability  to 
work  with  culture.” 

F.  NAVAL  AVIATION  AS  A  HIGH  RELIABILITY  ORGANIZATION 

Naval  Avaition  is  one  of  many  organizations  that  is  recognized  as  a  HRO.  As 
part  of  the  HFQMB’s  drive  to  reduce  human  error  in  aviation  mishaps,  Ciavarelli  and 
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Figlock  (1997)  created  a  survey  instrument,  the  Command  Assessment  Survery  (CAS). 
Based  upon  Libuser’s  MOSE,  the  survey  attempts  to  capture  and  assess  the  HRO  aspect 
of  Naval  Aviation  from  an  aircrewman's  perspective.  Based  on  the  analysis  of  1,254 
aviators  who  responded  to  the  survey  the  CAS’s  results  indicate  that  organizational  and 
supervisory  issues  are  seen  by  aircrewmen  as  impacting  flight  safety  (Ciavarelli  & 
Figlock,  1997). 

Baker  (1998)  modified  the  CAS  and  developed  a  prototype  67 -item  Maintenance 
Climate  Assessment  Survey  (MCAS)  in  an  attempt  to  capture  and  assess  safety  from  the 
perspective  of  Naval  Aviation  aircraft  maintainers.  With  the  assistance  of  some  experts 
in  the  field  of  aviation  maintenance  67  questions  were  initially  developed  from  155 
candidate  questions  that  were  tailored  specifically  to  aviation  maintenance.  One 
additional  maintenance  specific  component  was  created  and  added  to  the  existing  MOSE: 
Communication/Functional  Relationships.  Through  principal  component  analysis  Baker 
finds  that  his  survey  data  is  dominated  by  one  dimension  accounting  for  nearly  a  third  of 
the  data’s  variance  with  all  six  of  the  MOSE  components  loading  upon  the  dominant 
component.  Through  factor  analysis  and  a  number  of  other  regression  techniques  Baker 
is  then  able  to  successfully  reduce  the  prototype  67  question  survey  into  a  comprehensive 
35  question  instrument.  It  is  the  35  question  variant  of  the  MCAS  developed  by  Baker 
that  is  applied  in  this  study  to  further  improve  the  understanding  of  the  possible 
influences  of  human  factors  in  aviation  maintenance  operations  within  the  FLSW. 
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G.  SUMMARY 


Latent  accident-producing  conditions  are  present  now;  it  is  not  necessary  to  wait 
for  bad  events  to  find  out  what  they  are  argues  Reason  (1997).  Organizations  such  as 
Naval  Aviation  need  principal  ways  of  identifying  their  most  urgent  process  problems  in 
order  to  deploy  limited  resources  in  most  efficient  and  timely  manner.  The  MCAS  can  be 
one  such  tool.  Differences  identified  in  safety  climate  assessments  from  commonly  held 
perceptions  among  organizations  may  prove  to  be  a  root  source  of  certain  unsafe  attitudes 
and  behaviors.  The  early  detection  of  these  differences  will  undoubtedly  become  an 
integral  part  of  a  proactive  and  vital  mishap  prevention  process.  To  conclude,  the 
question  of  whether  one  can  effectively  measure  an  organization’s  safety  climate  and  take 
corrective  action  to  mend  deficient  processes  prior  to  the  occurrence  of  a  accident, 
continues  to  be  perhaps  the  most  important  issue  in  risk  management  today,  both  for 
Naval  Aviation  and  other  high-reliability  organizations. 
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III. 


METHODOLOGY 


A.  RESEARCH  APPROACH 

The  intent  of  this  study  is  to  assess  the  aviation  maintainer’s  perception  on  safety 
and  to  achieve  a  better  understanding  of  the  climate  within  the  organization  in  which 
he/she  operates.  This  study  involves  the  analysis  of  data  obtained  from  a  Maintenance 
Climate  Assessment  Survey  (MCAS)  that  is  based  on  an  existing  model  of  High- 
Reliability  Organizations  (HROs).  This  is  done  in  order  to  identify  factors  that  may  be 
utilized  in  the  improvement  of  safety  in  aviation  maintenance  practices.  The  data 
analysis  in  this  thesis  entails  descriptive  statistics,  principal  component  analysis,  cluster 
analysis,  analysis  of  variance  (ANOVA),  and  multiple  comparisons  in  ANOVA. 

B.  DATA  COLLECTION 

1.  Subjects 

The  subjects  used  in  the  data  collection  are  Navy  officers  and  enlisted  personnel 
involved  in  aviation  maintenance  of  the  Fleet  Logistic  Support  Wing  (FLSW);  no 
civilians  participated  in  the  survey.  The  survey  was  administered  to  13  of  the  14 
Squadrons  of  the  FLSW.  The  lone  squadron  not  surveyed  (VR-51,  Kaneoe  Bay,  HI) 
utilizes  civilian  contract  maintenance  of  its  aircraft;  meanwhile,  the  13  Squadrons  that 
were  surveyed  are  all  maintained  by  selective  reserve  and  active  duty  personnel. 
Additionally,  those  thirteen  squadrons  represent  three  different  communities  of  aircraft: 
the  C-9B  “Skytrain  n,”  the  C-130T  “Hercules,”  and  the  C-20  “Gulfstream.”  The  survey 
respondents  are  primarily  enlisted  Navy  personnel  assigned  to  aviation  maintenance 
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departments.  Listed  in  Table  1  are  the  participating  Squadrons  of  the  FLWS,  their 
locations  and  aircraft  type. 


SQDRN 

LOCATION 

AIRCRAFT 

VR-1 

WASHINGTON  D.C. 

C-20 

ATLANTA,  GA 

C-9 

WASHINGTON  D.C. 

C-20 

VR-52 

WILLOW  GROVE,  PA 

C-9 

VR-53 

WASHINGTON  D.C. 

C-130 

VR-54 

NEW  ORLEANS,  LA 

C-130 

VR-55 

MOFFIT  FIELD,  CA 

C-130 

VR-56 

NAS  NORFOLK,  VA 

C-9 

VR-57 

NORTH  ISLAND,  CA 

C-9 

VR-58 

JACKSONVILLE,  FL 

C-9 

VR-59 

FT  WORTH,  TX 

C-9 

VR-61 

WHIDBEY  ISLAND ,  WA 

C-9 

VR-62 

BRUNSWICK,  ME 

C-130 

Table  1.  FLSW  Participating  Squadrons. 


2.  Instrument 

The  Maintenance  Climate  Assessment  Survey  (MCAS)  developed  by  Baker 
(1998),  a  self-administered,  group  survey  consisting  of  fifteen  demographic  and  thirty- 
five  maintenance  related  questions  is  used.  The  demographic  portion  of  the  survey 
includes  questions  that  gather  responses  on  variables  such  as  respondent’s  rank,  service, 
work  shift,  years  of  service,  and  years  of  maintenance  experience.  It  also  includes 
inquiries  into  the  education  level,  rating,  age,  and  current  maintenance  qualifications  of 
the  survey  participants.  Anonymity  of  respondents  are  maintained  at  all  times.  The  35 
maintenance  related  questions  are  generated  from  the  five  MOSE  components  of  Process 
Auditing,  Reward  System,  Quality  Assurance,  Risk  Management,  Command  and  Control, 
and  the  aviation  maintenance  specific  component  of  Communication/Functional 
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Relationships.  The  survey  utilizes  a  Likert  type  five  point  rating  scale  with  verbal 
anchors  as  follows:  Strongly  Disagree,  Disagree,  Neutral,  Agree,  and  Strongly  Agree 
(The  35  item  MCAS  variant  is  included  in  Appendix  A). 

3.  Procedure 

The  survey  is  administered  on  site  and  in  a  group  setting  at  the  various 
participating  Squadrons.  Additionally,  the  survey  is  given  in  conjunction  with  a 
scheduled  maintenance  safety  presentation  on  human  factors  issues  in  aviation.  The 
Squadrons  are  in  various  stages  of  training  and  operational  tasking  at  the  time  of  the 
survey  being  administered.  Some  Squadrons  are  in  a  safety  standdown,  some  are  taken 
during  a  “drill”  weekend  while  still  other  Squadrons  are  in  “off-drill”  weekend.  Lastly, 
some  are  executing  a  normal  flight  schedule  routine  while  other  Squadrons  are  in  a  “no 
fly”  status  weekend.  The  varying  operational  taskings  that  the  Squadrons  are 
simultaneously  undertaking  during  the  administration  of  the  MCAS  account  for  much  of 
the  variance  in  the  number  of  surveys  collected  from  each  Squadron.  The  potential 
MCAS  respondents  are  briefed  on  the  survey  and  its  purpose  and  questions  that  arise 
pertaining  to  the  survey  are  answered  by  the  survey  administer.  The  surveys  are  then 
immediately  collected  upon  completion  to  allow  for  maximum  accountability. 

For  the  purpose  of  this  study,  two  squadron’s  responses  (VR-55  and  VR-57)  are 
transformed  from  the  67  item  MCAS  utilizing  the  mapping  indicated  in  Baker’s  (1998) 
thesis  to  the  current  35  item  MCAS. 
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C.  DATA  ANALYSIS 


1.  Data  Tabulation 

Survey  data  is  manually  entered  into  an  Excel  spreadsheet.  The  spreadsheet 
consists  of  rows  of  respondents  and  columns  of  survey  questions  (both  demographic  and 
MOSE  component).  Demographic  questions  record  mainly  bivariate  and  multivariate 
responses.  Component  question  results  are  coded  into  the  database  by  assigning  scores  of 
1, 2, 3, 4,  or  5  corresponding  to  the  Likert  scale  of  “Strongly  Disagree,”  “Disagree,” 
“Neutral,”  “Agree,”  and  “Strongly  Agree,”  respectively.  If  no  response  was  given  to  the 
question  the  word  “BLANK”  is  entered.  Two  questions  (questions  5  and  27)  in  the  35 
item  MCAS  are  negatively  anchored.  A  negatively  anchored  questions  seeks  to  solicit  a 
response  of  “Strongly  Disagree”  or  “Disagree”  if  systems  addressed  in  the  question  are 
functioning  in  a  favorable  manner.  (A  response  of  “Strongly  Agree”  to  a  negatively 
anchored  question  would  raise  issues  of  concern  to  the  subject  matter  of  the  question.) 

For  clarity  of  discussion  in  the  remainder  of  this  study  both  questions’  responses  have 
been  transposed  to  reflect  responses  as  if  the  questions  are  positively  anchored.  All  data 
is  then  imported  into  S-Plus  4.5  for  further  multivariate  statistical  data  analysis. 

2.  Statistical  Analysis 

Descriptive  non-parametric  analysis  is  conducted  on  the  survey  data  in  order  to 
describe  basic  and  general  information  about  the  demographic  and  question  results. 

Basic  summary  statistics  are  developed.  Some  results  include  distributions  of  survey 
respondents  by  age,  service,  education,  rank,  and  current  maintenance  qualifications. 
Additionally,  frequency  of  responses,  means,  and  standard  deviations  are  tabulated  for 
the  MOSE  components,  individual  questions,  and  aircraft  communities.  The  S-Plus  4.5 
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(Mathsoft,  1997)  program  is  used  for  multivariate  statistical  analysis.  Principal 
Component  and  Cluster  methodologies  such  as  agglomerative  nesting  and  divisive 
analysis  are  used  to  further  validate  the  stability  of  the  MCAS.  Analysis  of  variance 
(ANOVA)  and  multiple  comparisons  utilizing  Tukey’s  procedure  for  pairwise 
comparison  are  utilized  to  compare  the  effects  of  the  three-leveled  factor  aircraft 
community  and  the  six-leveled  factor  MOSE  component  have  on  the  mean  responses  of 
the  survey’s  respondents. 
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IV.  RESULTS 


A.  DESCRIPTIVE  STATISTICS 

1.  Sample 

Nearly  1000  Maintenance  Climate  Assessment  Surveys  are  collected  from 
military  personnel  serving  in  the  squadrons  that  comprise  the  Fleet  Logistics  Support 
Wing.  Of  these,  819  surveys  indicate  some  amount  of  aviation  maintenance  experience 
on  behalf  of  the  respondent.  The  analysis  of  these  819  surveys  will  be  addressed  in  the 
Results  section  of  this  thesis. 

2.  Demographics 

Descriptive  statistics  are  developed  for  the  survey  respondents.  A  demographic 
breakdown  according  to  respondent’s  rank  was  created  for  each  community’s  aviation 
maintenance  experienced  personnel  and  is  presented  in  Table  2.  Approximately  72%  of 
those  participating  in  the  MCA  Survey  are  personnel  serving  in  enlisted  ranks  of  E4-E6. 
This  seemingly  high  percentage  makes  intuitive  sense  given  that  the  majority  of  aviation 
maintenance  performed  and  immediate  supervision  provided  are  from  service  members 
of  this  rank  category. 


Respondent  Rank  Response  [ 

Blank 

E1-E3 

E4-E6 

E7- 

E9 

Officer 

Community 

# 

% 

# 

% 

# 

% 

# 

% 

# 

% 

Grand  Total 

Cl  30 

3 

i.i% 

23 

8.5% 

195 

71.7% 

22 

8.1% 

29 

10.7% 

272 

C20 

0 

0.0% 

4 

7.1% 

42 

75.0% 

9 

16.1% 

1 

1.8% 

56 

C9 

6 

1.2% 

35 

7.1% 

349 

71.1% 

67 

13.6% 

34 

6.9% 

491 

Grand  Total 

9 

1.1% 

62 

7.6% 

586 

71.6% 

98 

12.0% 

64 

7.8% 

819 

Table  2.  MCAS  Respondents’  Rank  By  Frequency  of  Response  and  Percentage. 
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Junior  enlisted,  senior  enlisted  and  officer  numbers  appear  to  be  in  proper  and 
similar  proportion  for  all  communities  as  well  as  can  be  seen  in  the  histogram  displayed 
in  Figure  4  below.  Notice  that  the  communities  with  the  most/least  senior  enlisted 
participants  have  the  least/most  officers.  This  makes  sense  in  that  higher  level 
supervision  will  be  provided  by  either  the  senior  enlisted  or  officer  ranks  with  an 
aggregate  of  approximately  20%  of  the  survey  respondents  coming  from  either  of  these 
two  rank  categories.  Histograms  of  other  descriptive  demographic  variables  such  as: 
Service  Type,  Education  Level,  Rating,  Current  Maintenance  Qualifications,  and  Age  of 
this  study’s  participants  are  located  in  Appendix  (B). 
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Figure  4.  MCAS  Respondents’  Rank  by  Percentage. 
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B.  MOSE  COMPONENT  STATISTICS 


Table  3  displays  the  six  MOSE  categories  and  respective  MCAS  questions  that 
pertain  to  each  component.  Summary  statistics  from  evaluating  the  MOSE  categories  are 
displayed  in  Tables  5  through  12.  Each  table  contains  the  mean  community  response  for 
each  of  the  questions  pertaining  to  a  particular  MOSE  component  and  an  overall 
community  MOSE  component  mean.  Standard  deviations  indicating  dispersion  from  the 
mean  value  of  each  question  are  computed  and  are  found  in  Appendix  (C).  Bar  charts  of 
responses  may  prove  insightful  in  helping  visualize  the  proportion  of  Likert  scale 
responses  (1  =  “Strongly  Disagree,”  2  =  “Disagree,”  3  =  “Neutral/Mixed,”  4  =  “Agree,” 
5  =  “Strongly  Agree”)  per  community  per  MOSE  component  and  are  also  found  in  this 
section. 


MOSE  Category 

Questions 

Process  Auditing 

1,  2,  6,  8 

Reward  System 

4, 13, 14, 25, 32 

Quality  Assurance 

3,  7, 12, 16,  22,  26,  28,  29,  30 

Risk  Management 

9, 15,24,31,34 

Command  &  Control 

11,17,18, 19, 23, 35 

Communication  /  Functional 
Relationships 

5, 10, 20, 21, 27, 33 

Table  3.  MOSE  Categories  and  Questions. 


1.  Process  Auditing 

The  first  MOSE  component,  Process  Auditing  (PA),  contains  four 


questions: 
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1 . )  My  command  has  a  dedicated  program  that  targets  individual  training 

deficiencies  and  ensures  the  uniform  enforcement  of  SOPs  among 
maintenance  personnel. 

2. )  My  command  monitors  maintainer  qualifications  and  support  equipment 

licensing. 

6.)  My  command  adequately  reviews  and  updates  safety  practices,  follows 
established  standards  and  maintains  quality  control,  ensuring  that  all 
maintainers  are  responsible  and  accountable  for  safe  maintenance. 

8.)  Medical  and  safety  staff  are  used  to  help  identify,  manage,  and  temporarily 
restrict  personnel  with  personal  issues  and  those  who  pose  a  risk  to  safe 
maintenance  in  this  command. 


All  questions  are  positively  answered  with  a  mean  range  from  3.17  to  4.37  across  all 
questions  with  community  means  ranging  from  3.64  to  3.90  (see  Table  4).  Additionally, 
the  PA  component  yields  the  highest  combined  component  mean  of  all  the  six  MOSE 
categories  with  a  mean  value  of  3.76.  Below  in  Table  5  are  the  responses  and  percentages 
of  responses  to  the  PA  component  questions  generated  by  the  aircraft  communities  that 
comprise  the  VR  Wing.  Additionally,  Figure  5  is  a  histogram  of  the  PA  component 
responses. 


Questions 

Component 

Mean 

Community 

1 

2 

6 

8 

C-130 

3.50 

4.00 

3.88 

3.17 

3.64 

C-20 

3.64 

3.96 

3.86 

3.25 

3.68 

09 

3.82 

4.37 

4.10 

3.33 

3.90 

Table  4.  Process  Auditing  Component  Means. 
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Likert  Scale  Responses 

1 

2 

3 

4 

5 

Community 

# 

% 

# 

% 

# 

% 

# 

% 

# 

% 

Grand  Total 

Cl  30 

30 

2.8% 

93 

8.7% 

261 

24.4% 

537 

50.2% 

148 

13.8% 

1069 

C20 

8 

3.6% 

17 

7.6% 

46 

20.5% 

121 

54.0% 

32 

14.3% 

224 

1  C9 

22 

1.1% 

114 

5.9% 

374 

19.4% 

933 

48.4% 

484 

25.1% 

1927 

Grand  Total 

60 

1.9% 

224 

7.0% 

681 

21.1% 

1591 

49.4% 

664 

20.6% 

3220 

Table  5.  Process  Auditing  Component  Responses  by  Frequency  and  Percentage. 


Figure  5.  Process  Auditing  Component  Responses. 


2.  Reward  System 

The  second  MOSE  component  is  Reward  System  (RS)  consisting  of  questions: 
4.)  Unprofessional  behavior  is  not  tolerated  in  the  maintenance  department. 

13. )  Safety  concerns  or  unsafe  hazards  associated  with  maintenance/flight  line 

operations  can  be  reported  without  fear  of  retribution  knowing  that  the  W/C, 
Q/A,  or  M/C  supervisors  will  address  and  manage  them  for  proper 
corrections. 

14. )  Violations  of  SOP,  NAMP  guidelines  or  other  procedures  are  discouraged  in 

this  command. 
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25.)  Violations  of  SOP,  NAMP  guidelines,  or  other  procedures  are  not  common 
in  my  command. 

32.)  Personnel  are  comfortable  telling  supervisors  about  personal  problems 
including  illness. 


All  questions  are  answered  favorably  by  all  communities  with  mean  question  responses 
ranging  from  3.37  to  4.13.  The  C-20  community’s  overall  component  mean  exceeds  3.80 


perhaps  indicating  that  a  good  reward  system  is  perceived  to  be  in  place.  The  RS 
component  also  yields  the  highest  combined  component  mean  (tied  with  PA)  with  a  mean 
value  of  3.76  (see  Table  6).  Figure  6  displays  the  frequency  of  responses  within  the  RS 
component. 


Questions 

Component 

Mean 

Community 

4 

13 

14 

25 

32 

C-130 

3.75 

3.76 

3.78 

3.52 

3.37 

3.64 

C-20 

3.77 

3.80 

4.13 

3.71 

3.65 

3.81 

C-9 

3.96 

4.00 

3.95 

3.56 

3.36 

3.77 

Table  6.  Reward  System  Component  Means. 
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Figure  6.  Reward  System  Component  Responses. 
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3.  Quality  Assurance 

The  third  MOSE  component.  Quality  Assurance  (QA),  consists  of  nine 
questions.  Those  questions  are: 

3.)  My  command  has  a  reputation  for  quality  maintenance  and  tool  control  is 
taken  seriously. 

7.)  QARs/  CDIs  and  Maintenance  Safety  Petty  Officer  are  sought  after  billets  in 
my  command. 

12.)  Inspectors  perform  all  required  actions  before  sign  off. 

16.)  My  supervisors  are  aware  of  individual  daily  workload  requirements  and 
recognize  safety  achievements  through  rewards  and  incentives. 

22.)  Maintainer  staffing  is  sufficient,  is  equally  worked  and  is  equally 
stressed  /fatigued  from  shift  to  shift. 

26.)  Proper  tools  and  equipment  are  available,  serviceable  and  used  and  I  am 
provided  adequate  resources  (time,  personnel)  to  accomplish  my  job. 

28. )  Required  publications  are  available,  current,  and  used. 

29. )  The  QA  division  is  respected  and  CDIs  /  QARs  routinely  monitor 

maintenance  evolutions  ensuring  that  maintenance  gripes  are  either 
corrected  or  addressed  prior  to  flight. 

30. )  Signing  off  PQS/JQRs/PARs  is  taken  seriously,  not  gun  decked  and 

maintenance  quality  is  as  high  on  detachments  as  it  is  in  homeport. 


All  questions  with  the  exception  of  question  22  are  given  favorable  responses  by  all 
communities.  Question  22  earns  unfavorable  marks  from  the  C-130  and  C-20 
communities  with  the  C-9  community  rating  it  “neutral”.  Mean  responses  from  the 
communities  for  question  22  range  from  2.61  through  3.00.  Thus,  question  22  identifies 
an  area  of  mutual  concern  within  the  VR  Wing  that  should  be  further  explored  for 
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intervention  efforts.  Another  question  of  interest  is  question  3  which  yields  the  highest 
means  for  all  communities  within  the  QA  component  ranging  from  3.93  to  4.37. 
Community  means  for  the  overall  QA  component  range  from  3.51  to  3.73  (see  Table  7.). 
Figure  7  displays  the  frequency  of  responses  within  the  QA  component. 


TTTTTO71 

3 

7 

12 

16 

0130 

4.06 

3.32 

389 

335 

020 

3.98 

3.36 

391 

341 

09 

4.37 

3.57 

4.12 

353 

Questions 

Corrponent 

22 

26 

28 

29 

30 

Mean 

261 

338 

3.66 

384 

3.47 

351 

263 

339 

3.82 

382 

368 

355 

3.00 

339 

3.77 

4.10 

37 7 

373 

Table  7.  Quality  Assurance  Component  Means. 
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Figure  7.  Quality  Assurance  Component  Responses. 


4.  Risk  Management 

Table  8  reflects  the  mean  values  of  responses  to  the  Risk  Management  (RM) 
MOSE  component,  which  is  comprised  of  five  questions.  Those  questions  are: 
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9.)  Based  upon  my  command’s  current  manning  and  assets,  it  is  not  over¬ 
committed. 

15.)  Supervisors  are  more  concerned  with  proper  aircraft  maintenance  than 
mission  completion  and  do  not  allow  cutting  comers  to  meet  operational 
commitments. 

24.)  Personnel  turnover  does  not  affect  my  command’s  ability  to  operate  safely. 

31.)  Safety  is  an  integral  part  of  this  command’s  maintenance 

planning/flight  line  operations,  where  QARs  are  helpful  and  the  QA 
division  is  not  “feared.” 

34.)  Maintainers  are  never  purposely  put  in  an  unsafe  situation  to  meet  the 
flight  schedule. 

Two  questions  are  of  particular  interest  and  concern  in  this  component.  Question  9  is 
unfavorably  responded  to  by  the  C-130  community  with  a  mean  response  of  2.71. 
Additionally,  both  the  C-9  and  C-20  communities  respond  marginally  with  means  of  3.10 
and  3. 16  respectively.  The  relatively  low  responses  to  this  question  across  the  board  are 
of  concern.  Perhaps  of  greater  concern  is  the  noticeable  difference  (approximately  0.4)  in 
the  mean  response  of  C-130  community  and  the  other  aircraft  communities  within  VR 
Wing.  Question  24  is  the  other  noteworthy  question  in  that  both  the  C-130  and  C-9 
communities  respond  unfavorably  with  mean  responses  of  2.88  and  2.93  respectively 
while  the  C-20  community  produces  a  nearly  neutral  responses  of  3.14.  The  C-130 
community’s  RM  component  mean  is  a  meager  3.28  while  both  the  C-20  and  C-9 
communities’  component  means  are  3.50.  Additionally,  the  RM  component  yields  the 
second  lowest  combined  component  mean  of  all  the  six  MOSE  categories  that  comprise 
this  survey  with  a  mean  value  of  3.43.  Figure  8  displays  the  frequency  of  responses 
within  the  RM  component. 
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Questions 

Component 

Mean 

Community 

9 

15 

24 

31 

34 

C-130 

2.71 

3.56 

2.88 

3.79 

3.45 

3.28 

C-20 

3.16 

3.63 

3.14 

3.89 

3.67 

3.50 

C-9 

3.10 

3.80 

2.93 

3.86 

3.83 

3.50 

Table  8.  Risk  Management  Component  Means. 


Figure  8.  Risk  Management  Component  Responses. 


5.  Command  &  Control 

The  fifth  MOSE  component.  Command  &  Control  (CC),  is  made  up  of  six 
questions.  Those  questions  are: 

11.)  Maintenance  control  is  effective  in  managing  all  maintenance  activities, 

coordinating  between  M/C,  W/C,  and  QA  prior  to  the  incorporations  of  TDs. 

17.)  Supervisors  communicate  command  safety  goals  programs  and 
procedures. 
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18. )  W/C  supervisors  are  respected  by  the  maintenance  chief/officer. 

19. )  Qualified  personnel  properly  supervise  all  maintenance  evolutions  and 

maintainers  are  briefed  on  the  potential  hazards  associated  with 
maintenance  activities. 

23.)  Multiple  job  assignments  and  collateral  do  not  adversely  affect 
maintenance. 

35.)  Safety  education  and  training  in  my  command  are  comprehensive  and 
effective  and  the  safety  department  is  respected  by  the  supervisors  and 
maintainers. 

The  mean  responses  to  the  questions  range  from  2.74  to  3.91  as  can  be  seen  in  Table  9. 
The  C-20  community’s  mean  of  2.96  to  question  18  is  unfavorable  while  the  respective 
C-130  and  C-9  community  responses  of  3.30  and  3.67  are  modestly  positive  in  nature. 
The  noticeable  and  sizeable  differences  in  responses  among  the  three  communities  may 
suggest  the  presence  of  differing  cultural  attitudes  within  the  maintenance  communities. 
Another  question  of  significant  interest  is  question  23.  Both  the  C-130  and  C-9 
communities  respond  unfavorably  with  mean  values  of  2.74  and  2.75  respectively  while 
the  C-20  community  records  a  mean  response  of  3.14.  The  relatively  low  rating  given  by 
all  communities  indicates  a  need  to  address  the  perceived  adverse  effects  on  maintenance 
by  exploring  intervention  strategies  to  help  mitigate  those  effects.  Figure  9  displays  the 
frequency  of  responses  within  the  CC  component. 
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Questions 

Component 

Mean 

Community 

11 

17 

18 

19 

23 

35 

C-130 

3.46 

3.68 

3.30 

3.57 

2.74 

3.63 

3.40 

C-20 

3.34 

3.61 

2.96 

3.48 

3.14 

3.60 

3.36 

C-9 

3.74 

3.89 

3.67 

3.75 

2.75 

3.91 

3.62 

Table  9.  Command  &  Control  Component  Means. 
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Figure  9.  Command  &  Control  Component  Responses. 


6.  Communication/Functional  Relationships 

Table  10  displays  the  last  MOSE  component,  Communication/Functional 

Relationships  (CF),  which  is  comprised  of  six  questions: 

5.)  My  command  has  no  problem  with  passdown  between  shifts.  (Positively 
anchored) 

10.)  Within  my  unit,  good  communication  flow  exists  up  and  down  the  chain  of 
command. 

20.)  My  supervisor  shields  me  from  outside  pressures  which  may  affect  my  work. 
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21.)  QARs  are  never  pressured  by  the  maintenance  supervisors  to  sign  off  a  gripe. 

27.)  Maintenance  Control  always  troubleshoots  aircraft  discrepancies.  (Positively 
anchored) 

33.)  I  feel  I  get  all  information  (internal  and  external)  required  to  perform  my  job 
safely,  and  feel  free  to  report  safety  violations,  unsafe  performance  or  other 
unsafe  behavior. 

Question  5,  yielded  an  unfavorable  response  of  2.63  from  the  C-20  community  while 
both  the  C-130  and  C-9  communities  produced  mean  values  of  3.05  and  3.23.  Question 
10  is  also  unfavorably  scored  by  the  C-20  community  with  a  response  mean  of  2.61. 
(Note:  One  squadron  of  the  C-20  community  recorded  very  unfavorable  mean  values  for 
questions  5  and  10  that  not  only  drove  the  community  means  lower  but  more  importantly 
flagged  these  issues  for  immediate  intervention  steps  within  that  particular  squadron.) 
Question  20,  although  not  unfavorably  responded  to,  yielded  only  slightly  better  than 
neutral  ratings  across  the  board.  Question  27  is  of  interest  because  the  C-9  community’s 
response  of  3.01  is  well  below  the  other  communities’  mean  responses  of  3.45  (C-130) 
and  3.64  (C-20).  The  CF  component  yields  the  lowest  combined  component  mean  of  all 
the  six  MOSE  categories  with  a  mean  value  of  3.27  and  thus  issues  contained  in  this 
component  would  serve  as  a  good  starting  point  for  mitigating  risk  in  aviation 
maintenance  operations  within  the  VR  Wing.  Figure  10  displays  the  frequency  of 
responses  within  the  CF  component. 
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Questions 

Component 

Mean 

Community 

5 

10 

20 

21 

27 

33 

C-130 

3.05 

3.01 

3.00 

3.10 

3.45 

3.69 

3.22 

C-20 

2.63 

2.61 

3.13 

3.38 

3.64 

3.73 

3.18 

C-9 

3.23 

3.49 

3.29 

3.40 

3.01 

3.97 

3.40 

Table  10.  Communications/Functional  Relationships  Component  Means. 
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Figure  10.  Communication/Functional  Relationships  Component  Responses. 

7.  MOSE  Summary 

Some  generalized  observations  can  be  made  in  analyzing  the  MOSE  component 
mean  responses.  The  two  MOSE  components  of  greatest  concern  as  identified  by 
aviation  maintenance  personnel  of  the  FLSW  while  participating  in  the  MCAS  are  CF 
and  RM.  The  focusing  intervention  efforts  in  those  two  areas  should  be  a  priority. 

Almost  across  the  board  the  C-9  community’s  mean  responses  to  questions  are  noticeably 
higher  than  those  of  both  the  C-130  and  C-20  communities.  Additionally,  the  culture  of 
the  C-130  maintenance  community  is  very  sensitive  to  questions  that  pertain  to  issues  of 
being  over-committed,  under-manned/under-staffed,  burdening  disproportionate 
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workloads,  the  eroding  effects  of  personnel  turnover,  and  the  negative  impact  of 
collateral  duties  acting  upon  maintenance  safety.  The  old  adage  of  accomplishing  more 
with  less  seems  to  have  worn  thin  within  the  C-130  maintenance  community.  Lastly,  the 
C-20  maintenance  community  seems  to  express  concern  with  coordination,  functional 
relationships,  and  communication  related  issues. 

C.  PRINCIPAL  COMPONENT  ANALYSIS 

Principal  Component  Analysis  (Hamilton,  1992)  is  performed  on  the  data  in  an 
attempt  to  identify  a  small  set  of  factors  that  account  for  most  of  the  variability  of  the 
data.  Utilizing  S-PLUS  4.5,  the  command  “princomp  (X,  na.action=na.omit,  cor  =  F)”  is 
used  in  this  analysis.  “X”  is  a  819  by  35  matrix  of  survey  responses.  The  code 
“na.action  =  na.omit”  is  required  in  order  for  S-PLUS  to  handle  cases  with  missing 
observations.  S-PLUS  omits  all  cases  with  missing  values  (coded  NA)  and  does  not  use 
them  in  the  performance  of  a  principal  component  analysis.  Ninety  such  cases  are 
present  and  thus  discarded  from  the  original  data  due  to  responses  missing  in  the  35 
maintenance  related  questions  that  comprised  the  second  part  of  the  survey.  This  reduces 
the  number  of  cases  to  be  evaluated  in  the  principal  component  analysis  to  729.  The  “cor 
=  F‘  implies  that  the  analysis  is  based  on  a  covariance  matrix  rather  than  a  correlation 
matrix.  The  covariance  matrix  is  chosen  since  the  original  observations  are  all  on  a 
common  scale.  A  scree-plot  displays  the  output  of  the  principal  component  analysis  (see 
Figure  1 1).  This  figure  displays  each  component’s  contribution  to  the  total  variance. 
Above  each  bar  in  the  plot  is  the  cumulative  percentage  of  variance  accounted  for  as 
successive  components  are  added.  Notice  that  nearly  one-third  of  the  data’s  variance  is 
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accounted  for  by  the  first  component  alone  with  the  next  nine  components’  diminishing 
contributions  accounting  for  only  another  third  of  total  variance.  This  suggests  that  the 
survey  data  is  dominated  by  the  first  component  dimension. 


x 


0.32 


Figure  11.  Principal  Component  Scree-Plot. 

The  principal  component  loadings  are  the  coefficients  of  the  principal  components 
transformation.  They  provide  a  convenient  summary  of  the  influence  of  the  original 
variables  on  the  principal  components  (S-Plus  4, 1997).  A  plot  of  the  variables  that  load 
on  the  first  five  components  is  generated  by  utilizing  the  following  S-PLUS  code: 
“plot(name$loadings),”  where  “name”  is  any  name  assigned  to  the  principal  component 
object.  The  dollar  sign  operator  ($)  attaches  the  “loadings”  property  of  the  principal 
component  object  for  plotting.  Figure  12  displays  the  first  five  components  and  the 
variables  (survey  questions)  that  load  upon  each.  (Note:  a  large  coefficient  (in  absolute 
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value)  corresponds  to  a  high  loading,  while  a  coefficient  near  zero  indicates  the  variable 
has  a  low  loading  on  the  component.) 
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Figure  12.  Principal  Component  Loadings. 


Identifying  the  six  questions  that  load  most  heavily  upon  the  most  dominant 
component.  Component  1,  reveals  that  four  of  the  six  MOSE  components  are  represented 
(CF,  RM,  CC,  and  RS).  By  further  looking  at  a  printout  of  the  principal  component 
loadings  it  is  evident  that  numerous  questions  load  quite  heavily  on  this  orthogonal 
projection  with  only  small  perturbations  in  their  values  (as  perhaps  can  be  inferred  by  the 
figure  above).  The  next  four  components  all  load  across  at  least  three  MOSE  components 
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with  Components  3  through  5  loading  on  the  QA,  CF,  and  RM  questions  in  particular. 
Thus,  it  cannot  be  said  that  just  any  one  or  two  MOSE  components  contribute  to  the 
majority  of  the  variance  within  the  survey  but  rather  that  all  six  MOSE  components  play 
an  integral  in  capturing  aviation  maintainers’  attitude  with  respect  to  safety. 

D.  CLUSTERING  ANALYSIS 

Clustering  analysis  is  a  way  of  classifying  a  data  set  into  groups  that  are  cohesive 
but  separate  (S-Plus  4,  1997).  For  this  study,  clustering  analysis  is  performed  on  the  35 
maintenance-related  questions  comprising  the  second  portion  of  survey  in  order  to  look 
for  trends  in  how  the  data  is  grouping.  Two  methods  by  which  this  can  be  achieved  are 
agglomerative  and  divisive  hierarchical  clustering.  Though  the  two  methods  attempt  to 
achieve  the  same  end  state,  the  partitioning  of  the  data  into  separate  groups,  their 
respective  algorithms  operate  in  polar-opposite  fashion. 

1.  Agglomerative  Nesting  (“agnes”) 

A  hierarchical  agglomerative  algorithm  constructs  a  hierarchy  of  clusterings. 
Considered  a  “bottom-up”  approach,  the  S-PLUS  agglomerative  nesting  algorithm  is 
called  “agnes.”  At  first,  each  observation  (survey  question)  is  a  small  cluster  by  itself. 
“Agnes”  then  yields  a  sequence  of  clusterings  whereas  the  two  "nearest"  clusters  are 
combined  to  form  one  larger  cluster  until  only  one  large  cluster  containing  all 
observations  remains.  The  S-PLUS  code  used  for  this  study  is  as  follows:  “agnes  (daisy 
(t  (X)),  diss=  T)”.  The  data  set  is  transposed  through  the  code  “t  (X)”  so  as  to  cluster 
across  the  35  survey  questions,  vice  the  cases.  The  S-PLUS  function  “daisy”  then 
computes  a  dissimilarity  matrix  for  the  transposed  data  matrix  “X.”  The  “diss=T” 
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parameter  implies  that  a  dissimilarity  matrix  will  be  used  in  the  algorithm’s  computation 
of  the  distance  function.  The  output  of  the  “agnes”  function  is  displayed  in  a  clustering 
tree  diagram  with  the  leaf  nodes  of  the  tree  being  the  survey  questions.  The  vertical 
coordinate  of  where  two  branches  join  on  the  clustering  tree  equates  to  the  dissimilarity 
between  combining  clusters.  The  command  “plot  (name)”  is  used  to  obtain  the  clustering 
tree,  where  “name”  is  any  name  assigned  to  the  “agnes”  created  object.  An  “agnes” 
clustering  tree  created  on  the  survey  data  is  shown  in  Figure  13. 


Figure  13.  “agnes”  Clustering  Tree. 

Additionally,  “agnes”  outputs  an  Agglomerative  Coefficient  (AC).  An  AC 
measures  the  amount  of  clustering  structure  of  the  data  set.  An  AC  of  one  (1)  would 
indicate  a  perfect  clustering  structure  and  a  zero  (0)  would  indicate  no  clustering 
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structure.  The  resulting  AC  produced  from  performing  “agnes”  on  the  data  set  is  0.38 
indicating  a  modest  clustering  structure  within  the  data  set. 

2.  Divisive  Analysis  (“diana”) 

A  divisive  hierarchical  algorithm  also  constructs  a  hierarchy  of  clusterings. 
Considered  a  “top-down”  approach,  it  begins  with  one  large  cluster  containing  all 
observations  and  divides  until  each  remaining  cluster  contains  only  a  single  observation. 
Clusters  with  the  largest  dissimilarity  between  any  two  of  its  observations  are  selected  to 
split  first.  To  divide  a  selected  cluster,  the  most  disparate  item  in  the  group  (i.e.,  the  one 
with  the  highest  average  dissimilarity  to  all  other  observations  in  the  selected  cluster) 
initiates  a  “splinter  group”  (Kaufman  &  Rousseeuw,  1990).  Then  for  each  item  outside 
the  splinter  group,  average  distances  are  computed  to  all  other  items  and  to  the  splinter 
group.  Items  that  are  closer  to  the  splinter  group  than  any  other  item  are  added,  otherwise 
it  is  paired  with  whichever  item  it  was  closest  to.  The  process  is  repeated  until  each 
cluster  contains  only  a  single  item. 

The  S-PLUS  code  used  to  perform  the  divisive  analysis  on  the  data  set  was 
“diana  (daisy  (t  (X)),  diss=  T).”  The  data  set  is  transposed  through  the  code  “t  (X)”  so  as 
to  cluster  across  the  survey  questions,  vice  the  cases.  The  S-PLUS  function  “daisy”  then 
computes  a  dissimilarity  matrix  for  the  transposed  data  matrix  “X.”  The  “diss=T” 
parameter  implies  that  a  dissimilarity  matrix  will  be  used  in  the  algorithm’s  computation 
of  the  distance  function.  The  output  of  the  “diana”  function  is  displayed  again  in  a 
clustering  tree  diagram.  The  command  “plot  (name)”  is  used  to  obtain  the  clustering  tree, 
where  “name”  is  any  name  assigned  to  the  object.  A  “diana”  clustering  tree  created  on 
the  survey  data  is  shown  in  Figure  14.  (Note:  Likert  scale  responses  to  both  questions  5 
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and  27  were  inverted  so  that  all  questions  considered  during  the  clustering  analysis  were 
positively  anchored.)  The  resulting  divisive  coefficient  produced  from  performing 
“diana”  on  the  data  set  is  0.46.  This  is  a  bit  larger  than  the  agglomerative  coefficient  of 
0.38  produced  by  “agnes”. 


Figure  14.  “diana”  Clustering  Tree. 

A  visual  comparison  of  both  “agnes”  and  “diana”  clustering  trees  reveals  very 
similar  clustering  results.  Though  there  is  no  distinct  clustering  by  MOSE  categories,  the 
items  do  group  similarly  when  both  agglomerative  and  divisive  hierarchical  clustering  is 
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performed  on  the  MCAS  data  set.  This  similar  grouping  of  the  survey  questions  coupled 
with  the  modest  agglomerative  and  divisive  coefficients  does  support  a  clustering 
structure  and  further  advocates  the  stability  of  the  survey. 


E.  ANALYSIS  OF  VARIANCE 

A  two-factor  analysis  of  variance  (ANOVA)  is  run  to  determine  if  either  the 
aircraft  community  factor  or  the  MOSE  component  factor  pose  any  effect  on  the  mean 
survey  response.  Let  the  letter  “7”  denote  the  number  of  levels  of  the  first  factor  of 
interest  (factor  “A”  or  community)  and  “7”  denote  the  number  of  levels  of  the  second 
factor  of  interest  (factor  “B”  or  MOSE  component).  With  7=3  (the  three  aircraft 
communities)  and  7=6  (the  six  MOSE  components),  there  are  then  7*7  =18  possible  cross 
classifications  consisting  of  a  level  of  factor  “A”  and  a  level  of  factor  “B”.  Each  such 
combination  is  viewed  as  a  treatment  and  is  denoted  by  Ky.  There  is  only  one  result  for 
each  possible  Ky  and  the  analysis  utilizes  a  two-factor  ANOVA  with  one  observation  per 
cell.  The  factors,  levels,  and  responses  are  shown  in  Table  1 1. 


PA 

RS 

QA 

RM 

CC 

CF 

C-130 

3.64 

3.64 

3.51 

3.28 

3.40 

3.22 

C-20 

3.68 

3.81 

3.55 

3.50 

3.36 

3.18 

C-9 

3.90 

3.77 

3.73 

3.50 

3.62 

3.40 

Table  11.  Community  Mean  Responses  per  MOSE  Component. 


There  are  two  hypotheses  of  interest  in  a  two-factor  ANOVA.  The  first  null 
hypothesis,  Hqa  ,  states  that  the  different  levels  of  factor  “A”  have  no  effect  on  the  true 
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average  response.  Likewise,  the  second  null  hypothesis,  H0b  ,  states  that  there  is  no 
effect  from  factor  “B”  on  the  true  average  response. 

The  information  in  Table  13  is  then  imported  from  an  Excel  spreadsheet  into  S- 
PLUS  4.5.  The  S-PLUS  code  for  conducting  the  ANOVA  is  “anova  ( aov  ( Avg  ~  factor 
A  +  factor  B,  data  =  X)),  where  “aov”  applies  the  ANOVA  function.  The  “anova”  at  the 
beginning  of  the  code  provides  degrees  of  freedom,  sum  of  squares,  and  mean  square 
values  as  well  as  the  F-test  statistics  and  corresponding  P-values  for  the  ANOVA  output. 
The  mean  responses  listed  in  Table  13  correspond  to  the  “Avg”  parameter  which  is 
modeled  by  each  aircraft  community  and  MOSE  component  (“factor  A”  and  “factor  B” 
respectively).  The  “X”  is  the  data  frame  that  contains  all  the  information.  The 
information  output  obtained  from  running  the  two-factor  ANOVA  is  displayed  below: 


Analysis  of  Variance  Table 

Response :  Avg 

Terms  added  sequentially  (first  to  last) 

Of  Sum  of  Sq  Mean  Sq  F  Value  Pr(F) 

Comm  2  0.1361446  0.0680723  13.31454  0.001516595 

Comp  5  0.5328003  0.1065601  20.84252  0.000053879 

Residuals  10  0.0511263  0.0051126 


With  an  alpha  level  of  0.05,  the  P-value  for  the  community  factor  variable  proves 
significant  at  0.001517.  This  results  in  a  rejection  of  Hoa  in  favor  of  the  alternate 
hypothesis,  which  states  that  the  communities  show  real  difference  with  respect  to  then- 
mean  responses  to  the  survey’s  MOSE  components.  The  MOSE  components  also  prove 
to  be  highly  significant  at  0.000054.  This  causes  us  to  reject  Hob  in  favor  of  the  claim 
that  the  different  MOSE  components  do  not  all  yield  the  same  mean  responses. 
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F.  MULTIPLE  ANOVA  COMPARISONS 


Since  both  factors  in  the  two-factor  ANOVA  reject  the  null  hypotheses  it  would 
be  interesting  to  find  out  which  of  the  levels  of  the  several  factors  are  significantly 
different  from  one  another.  A  multiple  comparison  procedure  such  as  Tukey’s  Procedure 
(or  the  “T”  method)  will  do  just  that.  Utilizing  the  Studentized  range  probability 
distribution,  confidence  intervals  for  all  pairwise  comparisons  are  computed  at  a  selected 
alpha  level.  In  the  first  case,  a  pairwise  comparison  of  the  three  aircraft  communities’ 
mean  responses  will  be  conducted.  The  resulting  confidence  intervals  represent  the  true 
values  of  all  differences  p*  -  Pj  between  true  treatment  means.  Each  interval  that  doesn’t 
include  the  value  of  zero  yields  the  conclusion  that  p*  and  Pj  differ  significantly  at  level  a 
level  of  alpha. 

The  S-PLUS  code  for  conducting  the  multiple  comparison  procedure  is 
“multicomp  (A,  focus  =  “factor”,  method  =  “tukey”,  plot  =  T).”  “multicomp”  is  the  call 
to  the  S-PLUS  function  that  performs  the  multiple  comparison  test.  “A”  is  the  object 
output  of  the  ANOVA  run  in  the  previous  section  on  the  information  from  Table  #.  The 
code  “focus  =  factor”  specifies  which  of  the  factors  to  plot  (in  this  case,  the  aircraft 
communities).  Various  methods  are  available  in  the  computation  of  the  comparisons,  but 
the  “method  =  “tukey””  code  explicitly  utilizes  the  Tukey  method.  The  function  provides 
the  standard  default  alpha  level  of  .05.  The  “plot  =  T”  code  automatically  prints  the 
confidence  intervals  to  the  graphics  device  and  are  displayed  in  Figure  15. 
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Figure  15.  Tukey’s  Pairwise  Comparison  of  Aircraft  Communities. 


The  S-PLUS  code  “tapply  (X  [/’response”],  X  [/’factor”],  mean))”  is  used  to 
generate  a  matrix  by  applying  the  “mean”  function  to  the  “response”  variable  (the  mean 
responses  listed  in  Table  #)  over  the  “factor”  levels  (the  aircraft  communities)  from  the 
data  frame  “X.”  Having  the  aircraft  community  means  of  3.45  (C-130),  3.51  (C-20)  and 
3.65  (C-9)  arranged  in  increasing  order,  the  graphical  display  of  the  confidence  intervals 
(Figure  15)  is  used  to  obtain  those  pairings  that  contain  the  value  of  zero.  Those 
components  were  then  underlined  as  shown  in  Figure  16.  Any  pair  of  aircraft 
communities  not  underscored  by  the  same  line  corresponds  to  a  pair  of  true  treatment 
means  that  are  significantly  different.  Items  within  the  same  grouping  do  not  differ 
significantly.  Thus,  Figure  16  shows  that  there  is  statistically  proven  difference  in  mean 
responses  between  the  C-9  community  and  both  the  C-130  and  C-20  communities  at  an 
alpha  level  of  .05. 
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Figure  16.  Identifying  Conununity  Means  That  Are  Statistically  Different. 

Now  a  multiple  comparison  procedure  is  run  on  the  other  significant  factor, 
MOSE  components,  and  its  graphical  output  is  displayed  below  in  Figure  17.  Once  again 
the  mean  value  responses  are  sorted  in  ascending  order  and  displayed  in  Table  12. 

Having  the  MOSE  component  mean  values  arranged  in  order,  the  graphical  display  of  the 
confidence  intervals  (Figure  1 1)  is  used  to  obtain  those  pairings  that  contain  the  value  of 
zero.  Those  components  were  then  underlined  as  shown  in  Figure  18.  Any  pair  of  MOSE 
components  not  underscored  by  the  same  line  corresponds  to  a  pair  of  true  treatment 
means  that  are  significantly  different  whereas  items  within  the  same  grouping  are  found 
not  to  differ  significantly.  Figure  18  shows  that  CF  is  found  to  be  significantly  different 
from  all  other  components  other  than  RM.  RM,  however,  is  only  significantly  different 
from  PA  and  RS.  Lastly,  CC  is  statistically  different  than  both  CF  and  RS  components. 
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Figure  17.  Tukey’s  Pairwise  MOSE  Component  Comparisons. 
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RS 

3.27 

3.43 

3.46 

3.60 

3.74 

3.74 

Table  12.  MOSE  Component  Means. 
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Figure  18.  Identifying  Statistically  Different  MOSE  Components. 
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V. 


CONCLUSIONS 


A.  FINDINGS 

The  results  of  this  study  conclude  that  the  MCAS  can  be  utilized  as  a  tool  for 
effectively  capturing  an  aviation  maintainer’s  perceptions  of  safety  in  maintenance 
operations.  Through  analysis  of  the  responses  a  number  of  different  questions  are 
identified  as  raising  concern  within  the  different  communities.  Those  questions  that 
yielded  unfavorable  responses  or  an  uncharacteristically  low  response  (question  27  for 
the  C-9  community)  are  recorded  in  Table  13  along  with  their  respective  MOSE 
component.  Discussions  of  the  individual  questions  are  included  in  the  MOSE 
Component  Analysis  section  of  Chapter  IV.  Overall,  the  general  safety  climate  of  the 
FLSW  is  good;  however,  some  potential  areas  within  each  community  have  been 
identified  for  focusing  and  prioritizing  safety  intervention  efforts. 


Questions  j 

CF 

RM 

CF 

CC 

CF 

QA 

CC 

RM 

CF 

5 

9 

10 

18 

22 

23 

24 

27 

C-130 

3.05 

2.71 

3.01 

3.30 

3.00 

2.61 

2.74 

2.88 

3.45 

020 

2.63 

3.16 

2.61 

2.96 

3.13 

2.63 

3.14 

3.14 

3.64 

09 

3.23 

3.10 

3.49 

3.67 

3.29 

3.00 

2.75 

2.93 

3.01 

Table  13.  Potential  Intervention  Areas  Identified  by  MCAS. 


Through  Analysis  of  Variance  (ANOVA)  and  Multiple  Comparison  testing  it  is 
shown  that  the  mean  responses  of  the  C-9  community  are  statistically  different  than  mean 
responses  of  both  the  C-130  and  C-20  aircraft  communities  that  comprise  the  FLSW 
based  upon  a  model  of  high  reliability  organizations.  Some  generalized  observations  can 
be  made  in  analyzing  the  MOSE  component  mean  responses.  Almost  across  the  board 
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the  C-9  community’s  mean  responses  to  questions  are  noticeably  higher  than  those  of 
both  the  C-130  and  C-20  communities.  Moreover,  the  culture  of  the  C-130  maintenance 
community  is  very  sensitive  to  questions  that  pertain  to  issues  of  being  over-committed, 
under-manned/under-staffed,  burdening  disproportionate  workloads,  the  eroding  effects 
of  personnel  turnover,  and  the  negative  impact  of  collateral  duties  acting  upon 
maintenance  safety.  The  old  adage  of  accomplishing  more  with  less  seems  to  have  worn 
thin  within  the  C-130  maintenance  community.  Additionally,  the  C-20  maintenance 
community  seems  to  express  concern  with  coordination,  functional  relationships,  and 
communication  related  issues.  Lastly,  statistical  significance  is  found  along  the  MOSE 
components  which  may  also  assist  in  helping  to  prioritize  intervention  efforts  within  the 
FLSW.  The  components  of  Cummunication/Functional  Relationships  and  Risk 
Management  pose  the  most  immediate  concern  to  the  aviation  maintainers  within  the 
FLSW  and  thus  would  serve  as  a  good  starting  point  for  mitigating  risk  in  aviation 
maintenance  operations. 

During  the  exploratory  phase  of  this  study,  a  few  questions  were  identified  for 
restructuring.  This  was  due  in  part  to  a  poor  fusing  of  questions  during  the  transition 
from  the  prototype  67  question  survey  to  the  35  question  survey  utilized  in  this  study.  A 
proposed  list  of  revised  MCAS  questions  has  been  produced  with  input  from  the  School 
of  Aviation  Safety,  AIRPAC’s  Maintenance  Training  Team,  and  the  Naval  Safety  Center 
and  is  included  in  Appendix  E. 
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B.  RECOMMENDATIONS 


This  study  can  be  used  as  a  starting  point  for  several  different  follow  on  studies. 
Ideally,  MCAS  should  now  be  incorporated  within  Naval  Aviation’s  regular  active  duty 
Wings  for  timely  identification  and  prioritization  of  potential  safety  intervention  areas 
within  aviation  maintenance.  Minor  modifications  of  the  MCAS  could  also  be  made  for 
a  Marine  Corps  specific  version  of  MCAS.  Further  analysis  and  comparisons  could  then 
be  made  among  similar  aircraft  communities  that  are  inherent  in  both  the  Navy  and 
Marine  Corps.  Additionally,  Marine  Corps  Reserve  aviation  units  could  then  be 
contrasted  with  this  study  of  the  Navy’s  FLSW.  Lastly,  a  more  in-depth  study  of  recent 
Mishap  Data  Analysis  and  MCAS  results  would  be  interesting  to  determine  to  what 
degree  there  is  correlation  in  perceptions  of  maintenance  safety  and  actual  safety  records. 
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APPENDIX  A.  35-ITEM  MCAS  QUESTIONNAIRE 


Purpose:  The  purpose  of  this  survey  is  to  try  and  gain  valuable  insight  into  the  maintenance  community’s 
perception  concerning  aviation  mishaps  within  the  Navy  and  Marine  Corps.  Your  participation  and  answers 
will  be  used  as  a  guide  in  the  Navy’s  on-going  efforts  to  lower  the  aviation  mishap  rate. 

The  first  fifteen  questions,  part  I,  regard  biographical  data;  information  particular  to  yourself.  This 
information  will  aid  in  the  analysis  of  your  responses.  NO  attempts  will  be  made  to  identify  individual 
respondents  or  their  organizations. 

Part  II  has  35  questions  pertaining  to  the  maintenance  community.  Please  respond  to  the  questions  with 
the  answer  that  most  correctly  reflects  your  honest  opinion.  Using  a  #2  pencil,  completely  darken  each 
response. 

Thank  you  in  advance  for  your  participation! 

PLEASE  RESPOND  TO  EACH  ITEM. 

1.  Your  rank?  (  )  E-l  -  E-3  (  )E-4  -  E-6  (  )CPO  E-7  +  (  ) Officer 

2 .  Your  community? 


VFA 

(  ) 

HS 

< 

) 

VMFA 

( 

) 

VF 

( 

) 

HSL  ( 

) 

VMA 

(  ) 

HC 

( 

) 

VP 

( 

) 

HCS 

( 

) 

VX  ( 

) 

VR 

(  ) 

VQ 

( 

) 

VAQ 

( 

) 

VAW 

( 

) 

3.  Your  designator?  (  LDO,  152X,  etc  )?  _ _  /  NEC  _ 

4 .  Are  you  currently  a  department  head?  (  )  Yes  (  )  No 

5.  Your  service?  (  )  USN  (  )  USNR  TAR  (  )  SELRES  (  )  Other 

6.  Your  shift?  (  )  DX  {  )  NX  (  )  MidX  (  )  Other _ 


7.  Total  years  of  service?  _ 

8.  Total  years  of  Aviation  Maintenance  experience?  _ 

9.  A-School  graduate?  (  )  Yes  (  )  No  (  )  N/A 

10.  Education  level : (  )  GED  (  )  High  School  (  ) College,  #yrs _ 

11.  Unit  home  location?  (  ) East  coast  (  )West  Coast  (  ) Other 

12.  Your  rating?  (  )  AD/AM  (  .)  AE/AT  (  )  PR/AME  (  )AO  (  )  Other 

13.  Your  age?  (  )  17-20  (  )  21-25  (  )  25-30  (  )  30+ 

14.  Current  maintenance  qualifications? 


{  ) Safe  for  Flight  ( 
(  )  CDI  ( 
(  )  SPO  ( 


15.  Duty:  (  )  Shore  {  )  Sea 


)  QAR 

)  Supervisor 
)  N  /A 


PartU 


Strongly  Agree  Neutral/  Disagree  Strongly 

Agree  Mixed  Disagree 


1 .  My  command  has  a  dedicated  program  that 
targets  individual  training  deficiencies  and  ensures 
the  uniform  enforcement  of  SOPs  among  maintenance 
personnel. 

2.  My  command  monitors  maintainer  qualifications 
and  support  equipment  licensing. 

3.  My  command  has  a  reputation  for  quality 
maintenance  and  tool  control  is  taken  seriously. 

4.  Unprofessional  behavior  is  not  tolerated  in  the 
maintenance  department. 

5.  My  command  has  a  problem  with  passdown 
between  shifts. 

6.  My  command  adequately  reviews  and  updates 
safety  practices,  follows  established  standards  and 
maintains  quality  control,  ensuring  that  all  maintainers 
are  responsible  and  accountable  for  safe  maintenance. 

7.  QARs/  CDIs  and  Maintenance  Safety  Petty  Officer  are 
sought  after  billets  in  my  command. 

8  .  Medical  and  safety  staff  are  used  to  help  identify, 

manage,  and  temporarily  restrict  personnel  with  personal 
issues  and  those  who  pose  a  risk  to  safe  maintenance  in 
this  command. 

9.  Based  upon  my  command’s  current  manning 
and  assets,  it  is  not  over-committed. 

10.  Within  my  unit,  good  communication  flow 
exists  up  and  down  the  chain  of  command. 

11.  Maintenance  control  is  effective  in  managing  all 
maintenance  activities,  coordinating  between  M/C, 
W/C,  and  QA  prior  to  the  incorporations  of  TDs. 


(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

1 2.  Inspectors  perform  all  required  actions  before 
sign  off. 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

1 3.  Safety  concerns  or  unsafe  ha2ards  associated 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

with  maintenance/flight  line  operations  can  be 
reported  without  fear  of  retribution  knowing  that  the 
W/C,  Q/A,  or  M/C  supervisors  will  address  and  manage 
them  for  proper  corrections. 


A.  Violations  of  SOP,  NAMP  guidelines  or  other  (  ) 

procedures  are  discouraged  in  this  command. 

(  ) 

(  ) 

(  ) 

(  ) 

15.  Supervisors  are  more  concerned  with  proper  aircraft  (  ) 
maintenance  than  mission  completion  and  do  not 

(  ) 

(  ) 

(  ) 

(  ) 

allow  cutting  comers  to  meet  operational  commitments. 
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Strongly 

Agree 

Agree 

Neutral/  Disagree 
Mixed 

Strongly 

Disagree 

16.  My  supervisors  are  aware  of  individual 

daily  workload  requirements  and  recognize  safety 
achievements  through  rewards  and  incentives. 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

17.  Supervisors  communicate  command  safety  goals, 
programs  and  procedures. 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

18.  W/C  supervisors  are  respected  by  the  maintenance 
chief/officer. 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

19.  Qualified  personnel  properly  supervise  all 
maintenance  evolutions  and  maintainers  are 
briefed  on  the  potential  hazards  associated  with 
maintenance  activities. 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

20.  My  supervisor  shields  me  from  outside 
pressures  which  may  affect  my  work. 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

21 .  QARs  are  never  pressured  by  the  maintenance 
supervisors  to  sign  off  a  gripe. 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

22.  Maintainer  staffing  is  sufficient,  is  equally  worked  and 
is  equally  stressed  /  fatigued  from  shift  to  shift. 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

23.  Multiple  job  assignments  and  collateral  duties 
do  not  adversely  affect  maintenance. 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

24.  Personnel  turnover  does  not  affect  my 
command’s  ability  to  operate  safely. 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

25.  Violations  of  SOP,  NAMP  guidelines,  or  other 
procedures  are  not  common  in  my  command. 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

26.  Proper  tools  and  equipment  are  available, 

serviceable  and  used  and  I  am  provided  adequate 
resources  (time,  personnel)  to  accomplish  my  job. 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

27.  Maintenance  Control  never  troubleshoots 
aircraft  discrepancies. 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

28.  Required  publications  are  available,  current, 
and  used. 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

29.  The  QA  division  is  respected  and  CDIs  /  QARs 
routinely  monitor  maintenance  evolutions  ensuring 
that  maintenance  gripes  are  either  corrected  or 
addressed  prior  to  flight. 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

30.  Signing  off  PQS/JQRs/PARs  is  taken  seriously, 
not  gun  decked  and  maintenance  quality  is  as  high 
on  detachments  as  it  is  in  homeport. 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

31.  Safety  is  an  integral  part  of  this  command’s 

(  ) 

(  ) 

(  ) 

(  ) 

(  ) 

maintenance  planning/flight  line  operations,  where 
QARs  are  helpful  and  the  QA  division  is  not  “feared”. 
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Strongly 

Agree 

Agree 

Neutral/ 

Mixed 

Disagree  Strongly 
Disagree 

32. 

Personnel  are  comfortable  telling  supervisors 
about  personal  problems  including  illness. 

(  ) 

(  ) 

(  ) 

(  ) 

( > 

33. 

I  feel  I  get  all  information  (internal  and  external) 
required  to  perform  my  job  safely,  and  feel  free  to 
report  safety  violations,  unsafe  performance  or  other 
unsafe  behavior. 

(  ) 

(  ) 

(  ) 

(  ) 

( > 

34. 

Maintainers  are  never  purposely  put  in  an 
unsafe  situation  to  meet  the  flight  schedule. 

(  ) 

(  ) 

(  ) 

(  ) 

( > 

35 

Safety  education  and  training  in  my  command  are 

(  ) 

(  ) 

(  ) 

(  ) 

< ) 

comprehensive  and  effective  and  the  safety  department 
is  respected  by  the  supervisors  and  maintainers. 
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APPENDIX  B.  DEMOGRAPHIC  VARIABLE  BAR  CHARTS. 


Figure  19.  MCAS  Respondent  Service  Type. 


60% 


BLANK  GED  ELS.  COLLEGE  COLLEGE 

3+YRS 


Figure  20.  MCAS  Respondent  Education  Level. 
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SAFE  FOR  SUPER-  QAR  SPO  CDI  N/A 

FLT  VISOR 


Figure  22.  MCAS  Respondent  Current  Maintenance  Qualifications. 
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I  I - 

BLANK  SEA  SHORE 


Figure  23.  MCAS  Respondent  Duty  Type. 


I  I - 1 - T 

17-20  21-24  25-29  OVER  30 


Figure  24.  MCAS  Respondent  Age. 
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APPENDIX  C.  SURVEY  QUESTION  STANDARD  DEVIATIONS 


Questions 

Community 

1 

2 

6 

8 

C-130 

0.96 

0.84 

0.77 

0.88 

C-20 

0.94 

0.83 

0.84 

0.98 

C-9 

0.86 

0.68 

0.75 

0.88 

Table  14.  Process  Auditing  Standard  Deviations. 


Questions 

Community 

4 

13 

14 

25 

32 

C-130 

0.97 

0.93 

0.92 

0.95 

1.05 

C-20 

0.95 

1.02 

0.84 

0.87 

1.08 

C-9 

0.93 

0.85 

0.91 

0.94 

1.06 

Table  15.  Reward  System  Standard  Deviations. 


Questions 

Community 

3 

7 

12 

16 

22 

26 

28 

29 

30 

C-130 

0.91 

0.96 

0.85 

1.02 

1.11 

1.06 

1.10 

0.83 

0.98 

C-20 

0.95 

1.00 

0.67 

1.17 

1.23 

1.11 

0.94 

0.79 

0.96 

C-9 

0.79 

0.92 

0.73 

0.99 

1.08 

1.06 

1.09 

0.74 

0.98 

Table  16.  Quality  Assurance  Standard  Deviations. 


Questions 

Community 

9 

15 

24 

31 

34 

C-130 

1.14 

1.03 

1.11 

0.86 

1.11 

C-20 

1.14 

1.10 

1.18 

0.83 

1.11 

C-9 

1.08 

0.91 

1.09 

0.90 

0.94 

Table  17.  Risk  Management  Standard  Deviations. 


69 


Questions  n 

Community 

11 

17 

18 

19 

23 

35 

C-130 

1.01 

0.85 

1.10 

0.87 

1.03 

0.90 

C-20 

1.05 

0.85 

1.16 

0.85 

1.05 

0.99 

C-9 

0.90 

0.80 

0.98 

0.85 

1.11 

0.83 

Table  18.  Command  &  Control  Standard  Deviations. 


Questions 

Community 

5 

10 

20 

21 

27 

33 

C-130 

1.07 

1.09 

1.00 

1.07 

1.08 

0.94 

C-20 

1.07 

1.17 

1.01 

1.07 

1.11 

1.11 

C-9 

1.05 

1.02 

0.97 

0.95 

1.16 

0.79 

Table  19.  Communication/Functional  Relationship  Standard  Deviations. 


APPENDIX  D.  PROPOSED  REVISED  MCAS  QUESTIONS 


COMPONENT  1:  PROCESS  AUDITING 

1.  The  command  adequately  reviews  and  updates  safety  practices. 

2.  The  command  monitors  maintainer  qualifications  and  has  a  program  that  targets 
training  deficiencies. 

3.  The  command  uses  safety  and  medical  staff  to  identify/manage  personnel  at  risk. 

4.  CDIs/QARs  routinely  monitor  maintenance  evolutions. 

5.  Tool  Control  is  taken  seriously  in  the  command  and  support  equipment  licensing  is 
closely  monitored. 

6.  Signing  of  PQS/JQRs/PARs  is  taken  seriously  and  not  gundecked. 

COMPONENT  2:  REWARD  SYSTEM  and  SAFETY  CLIMATE 

1 .  Our  command  climate  promotes  safe  maintenance  and  flight  operations. 

2.  Supervisors  discourage  SOP,  NAMP  guideline,  or  other  procedure  violations  and 
encourage  reporting  safety  concerns  without  fear  of  retribution. 

3.  Peer  influence  discourages  SOP,  NAMP  guideline,  or  other  procedure  violations  and 
individuals  feel  free  to  report  safety  violations,  unsafe  performance,  or  unsafe 
behaviors. 

4.  Violations  of  SOP,  NAMP  guidelines,  or  other  procedures  are  not  common  in  this 
command. 

5.  The  command  recognizes  individual  safety  achievement  through  rewards  and 
incentives. 

6.  Personnel  are  comfortable  approaching  supervisors  about  personal  issues/illness. 

7.  Maintenance  Safety  Petty  Officer,  Quality  Assurance  Representative,  and  Collateral 
Duty  Inspector  are  sought  after  billets  in  the  command. 

8.  Unprofessional  behavior  is  not  tolerated  in  the  command. 

COMPONENT  3:  QUALITY  ASSURANCE 

1.  The  command  has  a  reputation  for  quality  maintenance  and  has  set  standards  to 
maintain  quality  control. 

2.  The  QA  Division  and  Safety  Department  are  respected  in  the  command  and  are  seen 
as  essential  to  mission  accomplishment. 

3.  QARs/CDIs  perform  all  required  actions  before  sign-off  and  are  never  pressured  by 
maintenance  supervisors. 

4.  Maintenance  quality  on  detachments  is  the  same  as  that  at  home  station. 

5.  Required  publications/tools/equipment  are  available,  current/serviceable,  and  are 
exclusively  used. 

6.  QARs  are  viewed  as  helpful,  and  QA  is  not  “feared”  in  my  command. 
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COMPONENT  4:  RISK  MANAGEMENT 


1 .  Multiple  job  assignments  and  collateral  duties  adversely  affect  maintenance. 

2.  Safety  is  part  of  maintenance  planning,  and  additional  training/support  is  provided  as 
needed. 

3.  Supervisors  recognize  unsafe  conditions  and  manage  the  hazards  associated  with 
maintenance  and  flight  line  operations. 

4.  Iam  provided  adequate  resources  (time,  personnel  and  equipment)  to  accomplish  my 
job. 

5.  Personnel  turnover  does  not  negatively  impacts  the  command’s  ability  to  operate 
safely. 

6.  Supervisors  are  more  concerned  with  safely  conducting  aircraft  maintenance  than 
meeting  flight  schedule,  and  do  not  permit  cutting  comers  or  purposely  putting 
maintainers  in  unsafe  situations. 

7.  Maintainer  staffing  is  sufficient  from  shift  to  shift,  and  Day/Night  Check  have 
proportionately  stressful/fatiguing  workloads. 

8.  Supervisors  shield  personnel  from  outside  pressures  that  may  affect  their  work,  and 
are  aware  of  individual  workload  and  personal  issues. 

9.  Based  upon  my  command’s  current  manning/assets,  it  is  not  over-committed 

COMPONENT  5:  COMMAND  AND  CONTROL 

1.  My  command  temporarily  restricts  maintainers  who  are  having  personal  problems. 

2.  Safety  decisions  are  made  at  the  proper  command  levels,  and  CC  W/C  supervisors 
are  respected  by  the  maintenance  chief/officer. 

3.  Supervisors  communicate  command  safety  goals,  programs,  and  procedures,  and  are 
actively  involved  in  the  safety  program  and  management  of  safety  matters. 

4.  Supervisors  set  the  example  for  compliance  to  maintenance  standards  and  ensure 
uniform  enforcement  of  SOPs,  NAMP  guidelines,  and  other  procedures  among 
maintenance  personnel. 

5.  In  my  command,  safety  is  an  integral  part  of  all  maintenance  and  flight  line 
operations  and  all  maintainers  are  responsible  and  accountable  for  safe  maintenance. 

6.  Safety  education  and  training  in  my  command  are  comprehensive  and  effective. 

7.  All  maintenance  evolutions  are  properly  briefed,  supervised,  and  staffed  by  qualified 
personnel,  including  flight  line  activities  such  as  aircraft  moves. 

8.  Maintenance  control  is  effective  in  managing  all  maintenance  activities. 


COMPONENT  6:  COMMUNICATION  /  FUNCTIONAL  RELATIONSHIPS 

1 .  Good  communication  flow  exists  up  and  down  the  chain  of  command  and  I  get  all  the 
information  required  to  perform  my  job  safely. 

2.  Work  center  supervisors,  division  CPOs,  QA,  and  M/C  coordinate  their  actions, 
including  the  incorporation  of  TDs. 

3.  My  command  has  effective  pass-down  between  shifts. 
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4.  Maintenance  Control  always  troubleshoots  aircraft  discrepancies  and  gripes  are  either 
corrected  or  addressed  prior  to  flight. 

5.  Maintained  are  briefed  on  potential  hazards  associated  with  maintenance  activities. 
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